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Abstract 


Aspects  of  visual  alerts  were  explored  to  determine  what  type  of  visual  alerts  best  captured 
attention,  as  measured  by  reaction  time  (RT)  to  the  alerts.  In  part  1,  flash  rate  of  alerts  was 
explored  to  determine  the  most  effective  detection.  The  results  suggested  that  flashing  at  any  of 
the  flash  rates  was  no  better  than  a  non-flashing  alert.  In  part  2,  the  cost  of  moving  an  alert  from 
one  location  to  another  was  investigated.  The  results  showed  an  alert  location-switching  cost 
(slower  RTs  after  switch)  that  was  only  present  for  inexperienced  and  older  participants.  In  part  3, 
the  relationship  between  eye-location  and  cursor  location  was  explored.  The  goal  of  this  research 
was  to  determine  if  cursor-location  was  a  good  proxy  for  eye-location  during  a  multi-display  task 
that  had  been  used  in  previous  experiments.  The  results  suggested  that,  for  this  task,  the  cursor 
was  in  fact  an  excellent  proxy  for  eye -location  with  a  very  high  display  correspondence  between 
eye  and  cursor  location.  Both  eye  and  cursor  were  found  to  move  from  one  display  to  another  at 
the  same  time.  The  results  have  implications  for  future  visual  alerting  research  and  the  design  of 
automated  alerting. 


Resume 


Nous  avons  explore  certains  aspects  des  alertes  visuelles  pour  determiner  celles  qui  attiraient  le 
mieux  Tattention,  selon  la  mesure  du  temps  de  reaction  (TR)  aux  alertes.  Dans  la  partie  1,  le 
rythme  de  clignotement  des  alertes  a  ete  examine  afm  de  determiner  la  detection  la  plus  efficace. 
Les  resultats  ont  semble  indiquer  qu’aucun  des  rythmes  de  clignotement  n’etait  meilleur  qu’une 
alerte  non  clignotante.  Dans  la  partie  2,  nous  avons  etudie  le  cout  du  deplacement  d’une  alerte 
d’un  endroit  a  un  autre.  Les  resultats  ont  revele  qu’un  cout  associe  au  changement  d’ emplacement 
d’une  alerte  (TR  plus  longs  apres  le  changement)  n’etait  present  que  dans  le  cas  des  participants 
inexperimentes  et  plus  ages.  Dans  la  partie  3,  nous  avons  examine  la  relation  entre  la  direction  du 
regard  et  la  position  du  curseur.  Cette  recherche  avait  pour  but  de  determiner  si  la  position  du 
curseur  etait  un  bon  indicateur  de  la  direction  du  regard  au  cours  d’une  tache  sur  plusieurs  ecrans 
qui  avait  servi  lors  d’experiences  precedentes.  Les  resultats  ont  laisse  supposer  que,  pour  cette 
tache,  le  curseur  etait  en  fait  un  excellent  indicateur  de  la  direction  du  regard  et  que  la 
correspondance  entre  la  direction  du  regard  et  la  position  du  curseur  etait  tres  elevee  en  ce  qui 
conceme  les  ecrans.  Nous  avons  constate  que  les  yeux  et  le  curseur  allaient  d’un  ecran  a  l’autre 
en  meme  temps.  Les  resultats  ont  des  implications  pour  les  futures  recherches  sur  les  alertes 
visuelles  et  la  conception  d’alertes  automatisees. 
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Executive  summary 


Investigating  Visual  Alerting  in  the  Maritime  Domain:  Report  on 
3  experiments,  with  supporting  documentation 

Joshua  P.  Salmon;  Raymond  M.  Klein;  DRDC  Atlantic  CR  2010-058;  Defence 
R&D  Canada  -  Atlantic;  April  2010. 

Background:  In  command  and  control  centers,  like  the  operations  room  of  a  Navy  frigate,  where 
workload  is  high  and  decisions  critical,  automated  systems  that  can  assist  operators  are  a 
necessity.  One  such  system  would  be  an  automated  alerting  system,  implemented  as  a  means  of 
pointing  operators  to  critical  states  or  events  in  need  of  attention.  The  focus  of  the  research 
reported  here  is  primarily  aimed  at  alerting  operators  in  a  frigate  operations  room  using  visual 
alerting.  This  research  explored  a  number  of  questions  pertaining  to  visual  alerts,  and  it  extends 
recent  work  examining  the  use  of  automated  visual  alerting.  The  report  is  divided  into  three  Parts, 
each  addressing  different  issues  associated  with  alerts. 

Results:  In  Part  1,  flash  rate  was  explored  to  determine  whether  there  exists  an  optimal  flashing 
rate  for  visual  alerts  appearing  on  personal  desktop  computer  display  screens,  and  whether 
responses  to  flashing  alerts  are  faster  than  static  alerts.  No  difference  in  reaction  time  was  found 
as  a  function  of  flash  on-off  rate,  nor  whether  they  were  flashing  or  static,  although  reaction  time 
was  generally  slower  for  alert  detection  during  a  dual  task  as  compared  to  a  single  task.  In  Part  2, 
the  cost  of  switching  the  location  of  the  alert  was  investigated  during  a  multi-display  task.  This 
experiment  was  based  on  the  expectation  that  certain  kinds  of  visual  information  that  appear  in 
the  periphery  can  be  cognitively  packaged  with  central  vision  information  if  the  information  is 
task  relevant  and  if  the  location  is  consistent.  Thus,  it  was  anticipated  that  a  cost  in  response  time 
to  alerts  would  be  noted  when  the  location  of  the  alert  was  changed.  This  switching-cost  was  not 
found  for  all  participants,  but  did  appear  to  be  present  for  older  participants,  and  those  with  less 
experience  with  the  task.  In  Part  3,  the  relationship  between  eye -location  and  cursor-location  was 
tested.  The  goal  of  this  research  was  to  determine  the  extent  that  cursor  location  predicted  eye- 
location  during  a  multi-display  task.  The  results  indicated  a  strong  relationship  between  eye  and 
cursor  location  suggesting  the  cursor  is  a  good  proxy  of  eye-location  and  more  generally 
attention. 

Significance:  The  intended  application  of  this  research  is  toward  the  Halifax  Class  frigate 
operations  room  where  a  visual  alerting  system  could  support  the  current  auditory  alerting 
system.  The  results  of  these  three  studies  can  be  used  to  inform  the  design  of  automated  alerting 
systems  in  command  and  control  operations  centers.  Failure  to  identify  an  optimal  flashing  rate 
suggests  that,  for  the  task  used  in  this  work,  flashing  rate  is  not  important,  and  indeed,  this  result 
supports  previous  work  indicating  a  static,  non-flashing  alert  is  just  as  effective.  Moving  the 
location  of  the  alert  produced  variable  cost  in  alert  detection  depending  on  age  and  experience  in 
visual  tasks.  Thus,  these  factors  should  be  taken  into  account  in  the  design  of  visual  alerting 
interfaces. 

Future  plans:  Future  work  will  continue  to  investigate  attributes  of  visual  alerting  with  the 
objective  of  finding  the  most  appropriate  alert  types  for  complex  multi-display  environments. 
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Investigating  Visual  Alerting  in  the  Maritime  Domain:  Report  on 
3  experiments,  with  supporting  documentation 

Joshua  P.  Salmon;  Raymond  M.  Klein;  DRDC  Atlantic  CR  2010-058;  R  &  D  pour 

la  defence  Canada  -  Atlantic;  Avril  2010. 

Contexte  :  Dans  les  centres  de  commandement  et  de  controle,  comme  la  salle  des  operations 
d’une  fregate  de  la  Marine,  ou  la  charge  de  travail  est  lourde  et  les  decisions  cruciales,  des 
systemes  automatises  pouvant  aider  les  operateurs  sont  une  necessite.  Un  tel  systeme  serait  un 
systeme  d’alerte  automatise  mis  en  place  pour  signaler  aux  operateurs  les  etats  ou  evenements 
critiques  qui  requierent  leur  attention.  La  recherche  dont  il  est  question  ici  vise  principalement  a 
alerter  visuellement  les  operateurs  dans  la  salle  des  operations  d’une  fregate.  Elle  a  permis 
d’ explorer  un  certain  nombre  de  questions  liees  aux  alertes  visuelles  et  s’inscrit  dans  le 
prolongement  de  recents  travaux  portant  sur  l’utilisation  d’alertes  visuelles  automatisees.  Le 
rapport  est  divise  en  trois  parties,  chacune  traitant  de  differentes  questions  associees  aux  alertes. 

Resultats  :  Dans  la  partie  1,  le  rythme  de  clignotement  a  ete  examine  afm  de  determiner  s’il 
existe  un  rythme  optimal  pour  les  alertes  visuelles  qui  apparaissent  sur  les  ecrans  des  ordinateurs 
personnels  de  bureau,  et  si  les  reactions  aux  alertes  clignotantes  sont  plus  rapides  que  dans  le  cas 
des  alertes  fixes.  Nous  n’avons  decele  aucune  difference  dans  le  temps  de  reaction  en  fonction  du 
rythme  de  clignotement,  ni  aucune  difference  dans  le  fait  qu’une  alerte  soit  clignotante  ou  fixe; 
toutefois,  le  temps  de  reaction  etait  generalement  plus  long  pour  la  detection  des  alertes  durant 
une  double  tache  que  durant  une  tache  unique.  Dans  la  partie  2,  nous  avons  etudie  le  cout  du 
changement  d’emplacement  de  l’alerte  pendant  une  tache  executee  sur  plusieurs  ecrans.  Cette 
experience  reposait  sur  l’attente  selon  laquelle  certains  types  d’ information  visuelle  qui 
apparaissent  en  peripherie  peuvent  etre  regroupes  de  maniere  cognitive  avec  l’information  de  la 
vision  centrale  si  l’information  se  rapporte  a  la  tache  et  si  l’emplacement  demeure  constant. 
Ainsi,  nous  nous  attendions  a  noter  un  cout  au  niveau  du  temps  de  reaction  aux  alertes  lors  du 
changement  d’emplacement  de  l’alerte.  Or,  ce  cout  n’a  pas  ete  constate  pour  tous  les  participants, 
mais  il  a  semble  present  dans  le  cas  des  participants  plus  ages  et  de  ceux  qui  possedaient  une 
moins  grande  experience  de  la  tache.  Dans  la  partie  3,  nous  avons  examine  la  relation  entre  la 
direction  du  regard  et  la  position  du  curseur.  Cette  recherche  visait  a  determiner  dans  quelle 
mesure  la  position  du  curseur  predisait  la  direction  du  regard  durant  une  tache  executee  sur 
plusieurs  ecrans.  Les  resultats  ont  indique  une  solide  relation  entre  la  direction  du  regard  et  la 
position  du  curseur,  laissant  supposer  que  le  curseur  est  un  bon  indicateur  de  la  direction  du 
regard  et,  plus  generalement,  de  l’attention. 

Importance  :  L’ intention  consiste  a  appliquer  cette  recherche  a  la  salle  des  operations  de  la 
fregate  de  la  classe  HALIFAX,  ou  un  systeme  d’alerte  visuelle  pourrait  appuyer  le  systeme  actuel 
d’alerte  sonore.  Les  resultats  de  ces  trois  etudes  peuvent  servir  a  guider  la  conception  de  systemes 
d’alerte  automatises  dans  les  centres  de  commandement  et  de  controle.  L’incapacite  a  determiner 
un  rythme  de  clignotement  optimal  donne  a  penser  que,  pour  la  tache  utilisee  dans  le  cadre  de  ces 
recherches,  le  rythme  de  clignotement  n’est  pas  important.  En  effet,  ce  resultat  corrobore  des 
etudes  anterieures  indiquant  qu’une  alerte  non  clignotante,  c’est-a-dire  fixe,  est  tout  aussi 
efficace.  Le  fait  de  deplacer  l’emplacement  de  l’alerte  a  engendre  un  cout  variable  au  niveau  de  la 
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detection  des  alertes,  selon  Page  et  l’experience  des  personnes  executant  des  taches  visuelles.  Par 
consequent,  ces  facteurs  devraient  etre  pris  en  compte  dans  la  conception  d’interfaces  d’alerte 
visuelle. 

Projets  futurs  :  Les  travaux  futurs  continueront  de  porter  sur  l’etude  des  caracteristiques  des 
alertes  visuelles  et  viseront  a  trouver  les  types  d’alerte  les  plus  appropries  aux  environnements 
complexes  a  ecrans  multiples. 
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Figure  C-l:  Entire  set-up  from  a  distance  (calibration  step).  Participants  were  in  a  raised  chair 
in  front  of  faceLAB  cameras.  Experimenter  at  desk  in  front  of  laptops  (the 
‘experimenter’  in  this  photo  was  not  the  actual  experimenter  but  an  RA  posing  as 
the  experimenter) . 45 

Figure  C-2:  The  two  laptops  used.  LI  on  left  (controlled  faceLAB).  L2  on  right  controlled 
presentation  software  (Python).  Computer  in  back  (LCD)  was  used  for  data 
analysis  (but  not  used  during  the  experiment) . 45 
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1  Introduction 


This  research  was  completed  under  a  contract  between  Defence  Research  and  Development 
Canada  (DRDC)  Atlantic  and  Dalhousie  University,  Halifax,  Nova  Scotia.  The  write-up  is 
divided  into  three  parts  based  on  the  slightly  different  research  questions  each  part  was  aimed  at 
addressing. 

Part  1  investigated  flashing  alerts  and  flash  rates.  Specifically  it  sought  to  determine  what  flash 
rate  (if  any)  is  best  for  detection,  and  whether  or  not  flashing  alerts  are  more  salient  (can  be 
detected  more  quickly)  than  static  alerts.  This  research  also  explored  the  relationship  between  % 
of  time  alert-on  versus  %  of  time  alert-off,  and  explored  the  level  at  which  participants  perceive 
flashing  alerts  to  be  on  50%  of  the  time  and  off  50%  of  the  time.  This  research  was  conducted  at 
Dalhousie  in  the  Psychology  Department  (Dr.  Klein’s  lab). 

Part  2  investigated  the  cost  associated  with  alert  location  switching  during  a  multi-display  task. 
Specifically  participants  completed  30  minutes  of  a  task  with  visual  alerts  in  one  location 
followed  by  another  30  minutes  with  the  alert  switched  to  a  new  location.  The  goal  was  to 
examine  theories  of  spatial  attention  by  determining  the  cost  (in  terms  of  slowed  reaction  time) 
associated  with  having  to  respond  to  visual  alerts  in  a  new  location.  In  addition,  the  amount  of 
time  to  recover  (reaction  time)  after  a  switch  was  of  interest.  This  research  was  conducted  at 
DRDC-Atlantic. 

Part  3  investigated  eye -tracking  and  cursor  location  during  a  multi-display  task.  Specifically  this 
research  sought  to  investigate  the  assumption  that  during  this  task  the  participants’  eyes  are  on  the 
same  screen/display  as  their  cursor.  This  experiment  also  involved  changing  the  standard 
arrangement  of  the  displays  used  in  this  task  to  ensure  that  effects  of  screen  location  and  screen 
task  could  be  effectively  dissociated.  This  experiment  also  explored  a  number  of  other  research 
questions  including  whether  or  not  participants  move  their  eyes  to  a  new  display  before  the 
cursor,  and  whether  the  relationship  between  cursor  and  eye -location  changed  over  time.  This 
research  was  conducted  at  Dalhousie  in  the  Dalplex  (Kinesiology  Department). 

The  overall  goal  of  this  project  was  to  investigate  attributes  of  visual  alerting  in  order  to 
determine  what  types  of  visual  alerts  are  most  effective  at  capturing  attention  in  the  context  of 
multi-display  tasks.  The  overall  intended  application  of  this  research  is  toward  the  Halifax  Class 
frigate  operations  room  where  a  visual  alerting  system  could  support  the  current  auditory  alerting 
system.  Each  part  (1-3)  is  written  more-or-less  as  a  self-contained  article,  with  a  final  General 
Discussion  section  that  references  all  aspects  of  this  research. 

For  a  detailed  list  of  steps  involved  in  the  analyses,  and  explanation  of  the  data  files,  please  refer 
Annexes  A  and  B. 
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2  PART  1  -  Flash  Rate  Research 


Previous  research  at  Defence  Research  and  Development  Canada  (DRDC)  (Roberts  &  Foster- 
Hunt,  2008;  Roberts,  2008;  Crebolder  &  Beardsall,  2009a;  Crebolder  &  Beardsall,  2009b)  has 
considered  both  flashing  and  static  visual  alerts  with  the  expectation  that  a  flashing  alert  would  be 
more  salient  and  better  capture  attention.  However,  these  studies  have  failed  to  show  a  conclusive 
preference  for  flashing  alerts  over  static  alerts.  One  standard  flash  rate/period  (3.5Hz,  300ms)  was 
used  in  these  paradigms  and  the  objective  of  the  current  research  was  to  explore  whether  different 
flash  period  would  be  more  or  less  salient  than  the  standard  used  in  previous  research. 

This  research  involved  two  mini-experiments,  the  first  of  which  (Experiment  #1)  was  designed  to 
examine  the  perceived  on-off  ratio  (or  duty  cycle)  of  an  alert  -  for  example  whether  or  not 
participants  perceived  a  400  ms  period  visual  alert  shown  for  200  ms  on  and  200  ms  off  to  be  on- 
off  50%  of  the  time  (a  50/50  duty  cycle).  Research  conducted  by  Dr.  Klein,  and  visual  persistence 
literature  suggested  that  in  order  for  a  flashing  alarm  to  be  considered  to  have  a  50/50  duty  cycle, 
the  alarm  would  really  need  to  be  off  more  time  than  it  was  on  (Klein,  1983;  Sperling,  1960). 
Thus,  Experiment  #1  explored  the  psychophysics  of  this  relationship  by  allowing  participants  to 
adjust  the  time-on/time -off  rates  until  they  perceived  alerts  to  50/50  duty  cycle  for  each  period. 
The  second  experiment  (Experiment  #2)  then  used  these  same  flash  periods  in  a  reaction  time 
(RT)  experiment,  with  both  single  and  dual-task,  to  determine  if  alerts  at  some  flash  rates  were 
responded  to  more  quickly  than  others. 

2.1  Experiment  #1  -  Flash  Adjusting  Experiment 

As  stated  above,  the  goal  of  this  research  was  to  determine  at  what  point  participants  perceived 
flashing  alerts  of  different  periods  to  have  a  50/50  duty  cycle.  For  each  “trial”  participants  were 
asked  to  adjust  the  relative  amount  of  time -on,  time -off  time  until  the  flashing  appeared  to  be  on 
50%  of  the  time.  The  ultimate  goal  was  to  determine  if  perceived  50/50  duty  cycle  for  alerts  were 
responded  to  more  quickly  than  actual  50/50  duty  cycle  (a  question  that  was  explored  in 
Experiment  #2). 

2.1.1  Method 

2. 1.1.1  Participants 

Seven  adult  participants  participated  (3  males).  All  participants  were  English  speaking  with 
normal  or  corrected-to-normal  vision.  All  participants  were  compensated  for  their  time. 

2.1. 1.2  Stimuli 

Red  and  white  alerts  (10  cm  long  and  2  cm  tall)  were  shown  flashing  with  different  periods.  Five 
periods  were  chosen:  100  ms,  200  ms,  300  ms,  400  ms,  and  600  ms.  Alerts  were  shown  above  or 
below  a  central  fixation  at  one  of  two  distances  of  7.3  or  14.6  visual  angles.  Alerts  were  also 
shown  in  one  of  two  colors,  red  (RGB:  255,  0,  0),  and  white  (RGB:  255,  255,  255).  The  central 
fixation  was  a  grey  circle  (RGB:  150,  150,  150)  that  subtended  a  visual  angle  of  0.6  degrees  of 
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visual  angle  (0.5  cm).  In  summary,  there  were  four  variables  of  interest  (1)  period  (with  5  levels), 
(2)  location  (2  levels-top  or  bottom),  (3)  distance  (close,  near),  (4)  color  (white,  red).  These 
variables  were  shown  in  a  mixed  design  (not  blocked). 

The  experiment  was  run  on  an  iMac  (24”  LCD  screen)  running  OSX  on  2  GHz  Intel  Core  2  Duo 
processor.  Graphics  were  controlled  by  a  NVIDIA  GeForce  9400  card  with  256  MB  of  VRAM  at 
a  resolution  of  1920x1200,  and  refresh  rate  of  16.67  ms  (60  Hz).  Python  v2.6  with  pygame  was 
used  to  draw,  and  present  the  stimuli. 

2.1. 1.3  Procedure 

Prior  to  each  trial  the  starting  duty  cycle  of  an  alert  (percentage  on  to  alert  off  time)  was  randomly 
chosen.  For  example,  a  300  ms  (period)  alert  could  be  chosen  to  be  on  for  50ms,  and  off  for  250 
ms.  Participants  were  asked  to  adjust  the  percentage  of  alert  on  by  pressing  up  (more  on)  or  down 
(less  on)  on  a  keyboard.  Participants  were  instructed  to  adjust  the  rates  until  they  perceived  the 
alert  to  be  on  for  50%  of  the  time  and  off  for  50%  of  the  time  (50/50  duty  cycle).  Once  they  were 
satisfied  with  the  percentage  on/  percentage  off  they  would  advance  to  the  next  trial  by  pressing 
the  CTRL-key.  Participants  were  also  asked  to  wait  approximately  3  seconds  before  pressing  the 
CTRL -key  (to  ensure  confidence  in  their  decision).  Participants  were  asked  to  maintain  fixation 
during  rating.  Participants  sat  an  unrestrained  distance  of  approximately  45  cm  from  the  screen. 

To  become  familiar  with  the  task  participants  were  given  8  practice  trials  with  400,  and  600  ms 
periods.  For  the  full  experiment,  each  alert  was  shown  4  times,  for  a  total  of  160  trials.  The  entire 
experiment  lasted  between  30  minutes  to  1  hour  (depending  on  how  quickly  perceptual  judgments 
were  made). 

2.1.2  Results 

A  5x2x2x2  (period  x  color  x  location  x  distance)  mixed-effects  analysis  revealed  a  main  effect  of 
period,  F  (4)  =  679.73.  With  increased  period,  more  alert-on  was  required  to  perceive  a  50/50 
duty  cycle.  No  other  main  effects  or  interactions  were  significant.  This  suggested  that  the  only 
variable  affecting  the  perception  of  50%  on,  50%  off  was  the  period.  No  other  variables  (color, 
location,  or  distance  from  fixation)  had  an  effect. 

Notably,  for  all  participants,  less  time  on  was  required  to  perceive  50/50  duty  cycle  (see  Figure 

1). 
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Figure  1:  Periods  (100,  200,  300,  400,  and  600)  by  the  time  participants  reported  the  alert 
needed  to  be  on  (to  perceive  a  50/50  duty  cycle).  The  black  solid  line  shows  the  ACTUAL  50/50 
point  for  each  period.  Note,  all  participants  ’  values  were  below  the  actual  50/50  line. 

2.1.3  Discussion 

These  results  show  quite  clearly  that  in  order  to  perceive  a  50/50  duty  cycle  participants  actually 
need  the  alert  to  be  on  for  less  time  than  it  is  off.  Note,  this  pattern  was  shown  in  all  participants, 
although  the  amount  of  needed  on-off  did  vary  by  participant.  These  results  were  then  used  to 
explore  reaction  time  differences  to  flashing  alerts  in  Experiment  #2. 

2.2  Experiment  #2  -  Flash  Reaction  Time  (RT)  Experiment 

The  goal  of  Experiment  #2  was  to  use  the  periods  (flash  rates)  and  perceived  50/50  duty  cycle 
from  Experiment  #1  to  explore  what  types  of  alerts  are  responded  to  fastest.  This  was  explored  in 
both  a  single-task  (respond  to  alerts  only)  and  dual-task  (respond  to  alerts  while  concurrently 
doing  a  mouse-tracking  task)  condition.  Two  more  locations  were  included  in  this  design  -  left 
and  right  (in  addition  to  top  and  bottom).  Whether  or  not  an  experimenter  was  present  during 
testing  was  also  explored  as  a  variable.  This  variable  was  included  to  determine  whether  or  not 
having  an  experimenter  present  increased  arousal  and  motivation  of  participants  (in  the  form  of 
faster  responses  to  alerts).  The  general  goal  of  this  research  was  to  determine  what  types  of  alerts 
were  responded  to  fastest. 
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2.2.1 


Method 


2. 2. 1.1  Participants 

Twelve  adult  participants  participated  (3  males).  All  participants  had  normal  or  corrected-to- 
normal  vision.  Only  one  participant  reported  a  visual  anomaly  (a  slow  conducting  optic  tract  in 
one  eye),  although  the  results  of  this  participant  did  not  present  differently  than  those  of  the  other 
participants.  Participants  were  compensated  for  their  time. 

2.2.1.2  Stimuli 

Red  and  white  alerts  were  shown  at  one  of  8  locations  on  the  screen.  There  were  4  near  fixation 
locations  (approx.  7.3  visual  angles  from  fixation),  and  4  far  locations  (approx.  14.6  visual  angles 
from  fixation).  Vertical  alerts  (those  above  and  below  fixation)  were  drawn  10  cm  long  and  2  cm 
tall.  Horizontal  alerts  (those  to  the  right  and  left  of  fixation)  were  drawn  1 0  cm  tall  and  2  cm  wide 
(12.5  visual  angles  and  2.5  visual  angles,  respectively).  Alerts  were  shown  above  or  below  a 
central  fixation  at  one  of  two  distances  of  7.3  or  14.6  visual  angles.  Alerts  were  also  shown  in  one 
of  two  colors,  red  (RGB:  255,  0,  0),  and  white  (RGB:  255,  255,  255).  Central  fixation  was  a  grey 
(RGB:  150,  150,  150)  circle  that  subtended  0.6  visual  angles.  Fixation  subtended  a  visual  angle  of 
0.6  degrees. 

The  same  flash-rate  periods  from  Experiment  #1  were  used  (100,  200,  300,  400,  and  600).  Each 
period  was  set  so  that  the  percentage  of  the  time  on  corresponded  to  average  perceived  values 
from  Experiment  #1.  Specifically,  the  time  on  period  (in  refresh  rates)  were  chosen  to  be:  1,  3,  5, 
8,  &  14,  respectively.  This  corresponded  with  alert  on  times  (in  ms)  of:  16.67,  50,  83.33,  133.33, 
&  233.33.  Two  additional  periods  were  used.  The  first  was  a  300  ms  period  with  150  ms  alert-on 
time  (corresponding  to  the  REAL  50%  mark).  The  second  was  a  static  alert,  with  no  flash.  These 
were  labelled  as  300r  (real  50%)  and  600s  (static,  no  flash)  in  the  output  files.  All  other  alerts 
were  tagged  with  a  “p”  for  perceived  50/50  duty  cycle.  This  resulted  in  7  alerts  with  the  labels: 
lOOp,  200p,  300p,  400p,  600p,  300r,  and  600s. 

During  a  dual  task  block,  participants  controlled  a  hollow  black,  white-edged,  diamond  cursor, 
with  a  transparent  centre  that  subtended  a  visual  angle  of  approximately  0.6  visual  degrees 
(Figure  2). 

In  summary,  there  were  five  variables  of  interest  ( 1 )  period  (with  7  levels),  (2)  location  (with  4 
levels  -  top,  bottom,  right,  left),  (3)  distance  (close,  near),  and  (4)  color  of  alert  (white,  red),  (5) 
task  (single  or  dual). 

Experiment  #2,  like  Experiment  #1,  was  run  on  an  iMac  (24”  LCD  screen)  running  OSX  on  2 
GHz  Intel  Core  2  Duo  processor.  Graphics  were  controlled  by  a  NVIDIA  GeForce  9400  card  with 
256  MB  of  VRAM  at  a  resolution  of  1920x1200,  and  refresh  rate  of  16.67  ms  (60  Hz).  Python 
v2.6  with  pygame  was  used  to  draw,  and  present  the  stimuli. 
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2. 2. 1.3  Procedure 


Each  participant  participated  in  two  blocks  of  trials:  single  task  block  and  dual  task  block.  During 
the  single  task  block  participants  were  instructed  to  respond  to  alerts  (with  a  space  bar  press)  as 
quickly  as  possible  while  keeping  eyes  on  fixation.  Order  of  alert  presentation  was  randomized. 
Participants  sat  an  unrestrained  distance  of  approximately  45  cm  from  the  screen.  The  alert 
remained  on  the  screen  until  participants  responded.  Timing  of  alert  presentation  was  staggered 
using  a  Gaussian  noise  +1.5  seconds  to  help  eliminate  anticipation  responses  (that  is,  there  was  a 
minimum  1T1  of  1.5  s).  Anticipation  alerts  (false  positives)  were  recorded  /  flagged  with  a  500  ms 
grace  period.  That  is,  only  responses  that  occurred  500  ms  after  trial  onset  and  before  appearance 
of  an  alert  were  counted  as  false  positives.  In  false  positive  trials,  alerts  were  still  shown  until 
response,  and  reaction  time  of  response  (to  alert)  was  still  recorded. 

For  the  dual  task  condition  participants  used  the  mouse  to  keep  the  diamond-shaped  cursor  in  the 
middle  of  the  fixation  circle  (Figure  2).  Every  refresh  random  noise  was  added  to  the  location  of 
the  cursor  so  that  the  cursor  did  not  remain  stationary.  This  task  had  the  effect  of  providing  a 
continuous  distractor  of  a  consistent  difficulty  level.  It  also  encouraged  participants  to  maintain 
focus  around  the  fixation  mark.  In  the  dual  task  condition,  participants  continued  to  respond  to  the 
alerts  by  pressing  space  bar  with  their  other  hand.  Vertical  and  horizontal  cursor  distance  from  the 
fixation  was  recorded  at  each  refresh  and  saved  for  later  analysis.  Participants  were  instructed  to 
continue  responding  to  alerts  as  quickly  as  possible  while  performing  the  dual-task. 

All  participants  performed  the  mouse-task  with  the  hand  that  they  normally  used  to  control  a 
mouse  (for  all  but  one  participant,  this  was  the  right-hand).  That  is,  most  participants  responded 
to  alerts  with  their  non-dominant  (left)  hand,  with  the  dominant  (right)  hand  being  used  for  the 
mouse  task.  For  one  left-handed  participant,  their  non-dominant  (right)  hand  was  used  to  control 
the  mouse,  and  they  responded  to  alerts  with  their  dominant  (left)  hand  (this  was  because  they 
usually  use  the  mouse  in  their  non-dominant  hand).  Participants  used  the  same  hand  to  respond  to 
alerts  in  both  the  single  and  dual  task  conditions. 

Order  of  task  (single  vs.  dual)  was  counter-balanced  across  participants.  Each  participant 
completed  a  practice  block  prior  to  each  main  block.  The  main  blocks  lasted  approximately  1 6-20 
minutes,  and  consisted  of  336  trials.  The  entire  experiment  lasted  approximately  50  minutes. 
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Figure  2:  Examples  of  the  display  through  time.  Top  Panel  show  the  single  task  (Experiment#2 
single  task  or  Experiment  # 1 )  with  a  single  fixation  and  a  flashing  red  alert  near  the  fixation 
(top).  Bottom  Panel  shows  dual  task  with  the  cursor  (in  the  shape  of  a  diamond)  being  pushed 
around  the  screen  while  the  participant  attempts  to  maintain  the  cursor  at  fixation  (Experiment 
#2,  dual-task).  Alert  shown  in  Bottom  Panel  is  a  white  flashing  far  alert  on  the  left  side  of  screen. 

2.2.2  Results 

2. 2.2.1  Flash  Rate:  lOOp,  200p,  300p,  400p,  600p 

The  results  from  an  ANOVA  (5  levels  of  period)  suggested  no  difference  between  alerts  of 
different  flash  rates/periods,  F(A,7)  =  0.82,  p  =  .509.  The  means  RTs  for  all  flash  rates  were 
between  375  ms  and  379  ms  with  a  SD  =  73  ms. 

2. 2. 2.2  Perceived  50/50  (300p)  versus  Actual  50/50  (300r) 

The  results  from  a  second  ANOVA  (2  levels  of  period)  suggested  no  difference  between  an  alert 
shown  at  the  perceived  50/50  duty  cycle  versus  those  shown  at  the  actual  50/50  duty  cycle, 
A(l,10)  =  0.88, p  =  .347.  With  a  perceived  RT  =  379  ms,  and  an  actual-rate  RT  =  377  ms,  SD  = 
72  ms. 
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2. 2. 2. 3  Static  Alert  (600s)  versus  Flashing  Alert  (600p) 

Finally,  there  was  no  difference  between  static  and  flashing  alerts  in  this  experiment,  F(  1 , 1 0)  = 
0.11  ,p  =  .683.  With  a  mean  RT  of  376  ms  for  flashing  alerts  and  RT=374  ms  for  static  alerts,  SD 
=  70  ms. 

2. 2. 2. 4  Exploratory  Analyses:  Color,  Location,  Distance,  Task  &  Experimenter 

The  other  variables  (Color,  Location,  Distance,  Task,  and  Experimenter)  were  tested  in  a  2 
x4x2x2x2  ANOVA.  The  largest  main  effect  was  for  Task,  F(  1 )  =  917.11,  p  <  .001,  with 
participants  significantly  slower  at  responding  to  alerts  during  the  dual  task  condition,  M  =  353 
ms  compared  to  M=  400  ms.  There  was  also  a  main  effect  of  location,  F( 3)  =  6.82,  p  <  .001,  with 
top  and  bottom  bar  alert  RTs  faster  than  left  and  right  alerts,  M s=  372,  373  ms  compared  AT s= 
379,  382  ms,  respectively. 

Only  one  interaction  was  significant  (at  a=  .05)  and  that  was  the  interaction  between  task  and 
distance,  F(l)  =  4.51  ,p<  .05.  Specifically  these  results  showed  a  distance  effect  during  the  single 
task  (M=  356  ms  for  Far  alerts,  and  A7=35 1  ms  for  Near  alerts)  but  not  during  the  dual  task  (M= 
399  ms  for  Far  alerts,  and  M=400  ms  for  Near  alerts).  In  other  words,  distance  of  the  alerts  had  a 
bigger  effect  when  participants  did  not  have  the  mouse  dual-task  to  attend  to. 

2. 2. 2. 5  Mouse  Error  Analysis 

Mouse  error  lag  after  onset  of  each  alert  was  examined  for  1000  ms  post-alert  onset.  In  other 
words,  the  distance  of  the  cursor  to  the  fixation  (mouse  error)  was  examining  exactly  at  the  time 
an  alert  appeared  on  screen,  and  1000  ms  afterwards.  It  was  hypothesized  that  tracking  error 
would  increase  or  decrease  after  onset  of  alerts.  The  results  indicated  no  systematic  relationship 
between  performance  (error)  on  the  mouse-tracking  (secondary  task)  as  a  function  of  when  the 
alerts  appeared  during  the  primary  task. 

2.2.3  Discussion 

The  results  of  this  experiment  were  consistent  with  previous  studies  conducted  at  DRDC  (Roberts 
&  Foster-Hunt,  2008;  Roberts,  2008;  Crebolder  &  Beardsall,  2009a;  Crebolder  &  Beardsall, 
2009b),  showing  no  preference  for  flashing  alerts  compared  to  static  alerts.  In  addition,  there  were 
no  flash-rates/periods  that  appeared  to  capture  attention  better  than  others,  and  a  perceived  50/50 
duty  cycle  appeared  to  be  no  better  than  alerts  flashing  at  a  regular  50/50  duty  cycle.  These  results 
suggest  that  a  flashing  alert  is  no  better  or  no  worse  than  a  static  alert  during  these  tasks. 

The  additional  analyses  also  suggested  that  alerts  shown  at  the  top  and  bottom  of  the  screen  might 
be  better  at  attracting  attention  than  alerts  shown  at  the  side,  and  distance  of  the  alerts  may  have 
an  effect  (although  the  effect  appears  to  minimal).  Also,  colour  of  the  alert  does  not  seem  to 
matter  (at  least  white  does  not  appear  to  be  any  better  than  red  or  vice  versa),  and  presence  of  an 
experimenter  does  not  appear  to  affect  performance  on  these  tasks.  By  far,  the  biggest  predictor  of 
RTs  in  this  experiment  was  whether  or  not  participants  were  performing  the  dual-task.  All 
participants  were  slower  to  respond  to  alert  RTs  when  they  were  performing  a  concurrent  mouse- 
task. 
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2.2.4  Recommendations 


These  results  suggest  flashing  alerts  are  no  better  or  worse  than  static  alerts.  Therefore,  it  does  not 
appear  to  matter  what  types  of  alerts  are  used.  If  a  flashing  alert  is  used,  the  flash  rate  does  not 
appear  to  matter.  Additionally,  alerts  on  the  top  and  bottom  of  the  display  appear  to  be  detected  a 
little  faster  than  those  on  the  sides  of  the  screen. 
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3  PART  2  -  Location  Switch  Experiment 


This  study  was  conducted  at  DRDC  Atlantic  as  a  follow  on  to  previous  work  (Roberts  &  Foster- 
Hunt,  2008;  Roberts,  2008;  Crebolder  &  Beardsall,  2009a;  Crebolder  &  Beardsall,  2009b). 
Existing  software  programming  using  EPrime  Psychology  Software  was  used  as  the  basis  for  the 
experiment  with  the  modifications  outlined  in  Annex  A. 


3.1  Attentional  Limitations  in  Visual  Alerting 

3.1.1  Abstract 

A  study  was  conducted  to  investigate  the  switching  cost  of  changing  the  location  of  a  visual  alert 
while  participants  performed  a  high  intensity,  multi-display  task.  Based  on  the  proposition  that 
the  spatial  window  of  attention  can  be  extended  to  include  relevant,  though  non-task-related 
information,  it  was  hypothesized  that  response  times  to  the  alert  would  increase  immediately 
following  a  change  in  location  and  then  recover.  Generally,  results  showed  that  this  was  not  the 
case,  but  instead  response  time  increased  several  minutes  after  the  change  in  location  and  then 
recovered.  Further  investigation  revealed  that  age  and  expertise  (defined  as  experience  with  tasks 
involving  multiple  displays  or  video  gaming),  were  strong  moderators  of  the  effect  of  slowed 
response  after  switching.  Older,  less  experienced,  adults  showed  an  immediate  and  significant 
cost  that  was  not  shown  at  all,  or  was  shown  later,  by  the  younger,  more  experienced  adults.  The 
results  suggest  that  experience  with  a  specific  task,  or  more  general  video  game  experience,  can 
guard  against  the  cost  associated  with  moving  an  alert  to  a  new,  relatively  untrained  location. 

3.1.2  Introduction 

In  demanding  environments,  where  workload  is  high  and  outcome  critical,  operators  may  rely  on 
information  and  data  feeds  from  a  number  of  different  sources  to  aid  in  making  decisions  quickly 
and  accurately.  Many  command  and  control  centers,  like  the  operations  room  of  a  Navy  frigate, 
are  high  workload,  and  complex  environments,  where  multi-tasking  while  attending  to  several 
information  displays  is  all  too  familiar.  In  this  kind  of  intense  setting,  automated  systems  that  can 
assist  operators  are  a  necessity.  One  such  system  would  be  an  automated  alerting  system, 
implemented  as  a  means  of  pointing  operators  to  critical  states  or  events  in  need  of  attention. 

One  of  the  initial  challenges  facing  the  design  of  such  an  alerting  system  is  determining  the  mode 
of  delivery,  since  typically  both  visual  and  auditory  modalities  are  already  overburdened  in  fast- 
paced  environments.  The  focus  of  the  research  reported  here  is  primarily  aimed  at  a  frigate 
operations  room  and,  due  to  the  current  nature  of  that  environment,  the  visual  system  has  been 
selected  as  the  most  appropriate  avenue  for  further  investigation. 

Placing  alerts  in  the  display  periphery,  to  reduce  the  likelihood  that  the  stimulus  form  will 
obstruct  or  interfere  with  task-related  information,  has  been  identified  as  a  key  criterion. 
However,  visual  acuity  declines  with  retinal  eccentricity  making  information  in  the  periphery 
more  difficult  to  detect  (Engel,  1971).  Furthermore,  since  an  operator’s  task  often  necessitates  a 
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high  level  of  concentration,  focused  on  a  select  area  of  the  display,  it  is  not  unusual  that  important 
information  is  missed  if  it  appears  outside  or  in  the  periphery  of  the  area  of  focal  attention. 
Otherwise  known  as  attentional  tunnelling,  or  attentional  spotlight  (Wickens,  2005),  this  problem 
is  compounded  as  display  size  increases,  or  when  multiple  displays  are  used. 

In  general,  designing  an  effective  visual  alert  must  ensure  adequate  salience  so  a  stimulus  is 
effectively  noticed  by  operators  who  are  otherwise  engaged,  while  being  subdued  in  the  sense  that 
its  presence  is  not  intrusive  or  visually  distracting.  In  that  vein,  stimulus  form,  as  an  aspect  related 
to  attentional  salience,  has  been  examined  in  previous  studies  (Roberts  &  Foster-Hunt,  2008; 
Roberts,  2008;  Crebolder  &  Beardsall,  2009a;  Crebolder  &  Beardsall,  2009b). 

One  might  assume  that  the  larger  the  surface  area  a  stimulus  occupies,  the  more  likely  it  is  to  be 
successful  in  capturing  attention.  Somewhat  surprisingly  then,  previous  research  has  consistently 
shown  that  a  relatively  small  stimulus,  a  short  bar  (10  centimetres  (cm)  x  2  cm)  was  detected 
faster  than  a  larger  one,  a  border,  the  same  width  as  the  bar,  that  covered  the  entire  surrounding 
perimeter  of  the  display  (Roberts  &  Foster-Hunt,  2008;  Roberts,  2008;  Crebolder  &  Beardsall, 
2009a;  Crebolder  &  Beardsall,  2009b). 

Peripheral  vision  is  sensitive  to  temporal  changes  in  contrast  that  help  direct  attention,  and  the 
visual  system’s  orienting  mechanism  is  designed  to  point  attention  toward  eye-catching  stimuli 
(Coren,  Ward  &  Enns,  1994).  Cues  that  inform  us  about  where  something  will  likely  happen  are 
called  information  cues,  and  one  possibility  worth  considering  is  that  the  bar  is  a  more  effective 
information  cue  than  a  border  surrounding  the  edge  of  a  display.  Due  to  its  more  edged  and 
compact  form,  the  physical  appearance  of  a  bar  may  make  it  more  effective  at  directing  and 
capturing  covert  attention.  Posner  (1980),  by  demonstrating  the  effect  of  expectation  on  response 
time,  showed  that  attention  could  be  covertly  oriented  without  eye  movement.  In  fact,  the  area  of 
attentional  coverage  can  be  expanded  to  include  specific  locations  if  pre-knowledge  is  provided 
about  the  location  of  an  upcoming  stimulus  (Engel,  1971).  Thus,  the  spotlight  of  attentional  focus 
may  be  whole-task  specific,  expandable  to  include  associated  information,  even  if  that 
information  is  related  to  a  second,  unrelated  task.  If  this  were  so,  one  might  expect  a  cost  in 
detection  rate  if  the  location  of  the  stimulus  containing  the  information  changes.  This  study  was 
designed  to  examine  that  proposition,  using  a  short  bar  as  an  alerting  stimulus  presented  while 
individuals  performed  a  high-intensity  task. 

3.1.3  Method 

3. 1.3.1  Materials 

A  multi-display  workstation  consisting  of  three  21”  LCD  displays  (Samsung  SyncMaster  214T) 
was  used  to  present  the  task,  which  was  written  using  E-prime  Psychology  software.  The 
workstation  and  task  were  designed  to  emulate  a  high  intensity  workload  environment,  such  as 
the  one  a  naval  operations  room  operator  might  be  exposed  to.  Participants  were  required  to 
classify  and  report  targets  using  the  three  displays. 
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Figure  3:  Workstation,  showing  the  red  alert  in  the  bottom  location. 

Figure  3  shows  the  workstation.  The  centre  display  was  a  tactical  display  that  depicted  the 
participant's  ownship  in  the  centre,  and  incoming  ships,  represented  as  triangles,  moving  in 
incremental  steps  toward  the  ownship.  The  left  display  was  a  status  display  that  showed 
information  pertaining  to  selected  ships  (selected  by  mouse-over)  that  could  be  used  to  classify 
ships  as  ‘hostile’  or  ‘neutral’.  The  right  display  was  a  reporting  display  where  participants 
reported  ‘hostile’  ships  by  typing  one  letter  string  (‘asd’)  and  ‘neutral’  ship  with  another  letter 
string  (‘qwe’). 

Red  alerts,  10  cm  long  and  2  cm  wide,  were  displayed  periodically  (approx.  5.33  alerts/min, 
which  equalled  16  alerts  per  block)  on  all  three  displays  simultaneously.  Alerts  appeared  centered 
at  either  the  top  or  bottom  of  the  displays.  Duration  of  the  alerts  was  4  seconds,  or  until  a 
spacebar  response  was  made. 

3. 1.3.2  Participants 

Seventeen  male  and  7  female  volunteers  participated  in  this  study.  Participants  ranged  in  age  from 
20  to  60  (with  a  mean  age  of  42).  All  volunteers  reported  normal,  or  corrected  to  normal  vision;  2 
were  left-handed,  1  ambidextrous.  The  study  took  about  90  minutes  to  complete  and  participants 
were  reimbursed  following  DRDC  guidelines  (Keefe,  2010). 

3. 1.3. 3  Procedure 

Each  volunteer  participated  in  three  3-minute  practice  blocks,  and  twenty  3-minute  experimental 
blocks.  Participants  were  instructed  that  their  primary  task  was  to  respond  to  the  red  alerts  (with  a 
spacebar  press)  as  quickly  as  possible.  Their  secondary  task  was  to  identify  “vessels”  on  the 
tactical  display  by  using  the  information  that  appeared  on  the  status  display  when  a  vessel  was 
selected.  Correctly  identified  vessels  were  removed  from  the  display.  Motivation  for  the 
secondary  task  was  influenced  through  the  penalty  of  showing  an  image  of  an  exploding  ownship 
on  the  screen  whenever  vessels  came  within  the  predetermined  radius  of  the  ownship.  No  reward 
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or  penalty  was  given  for  responses  to  the  alerts,  but  the  instructions  did  emphasize  these 
responses  were  of  primary  importance. 

Two  alert  locations  were  possible,  top  and  bottom  of  the  displays.  For  the  first  ten  blocks  all  alerts 
were  shown  in  the  same  location.  For  half  the  participants  these  alerts  were  shown  on  the  top  of 
the  displays,  and  for  the  other  half,  the  bottom  of  the  displays.  On  the  eleventh  block  alert 
location  was  switched  (to  top  or  bottom). 

3.1.4  Results 

3. 1.4.1  Experience  Measure 

Because  of  the  skill  required  to  perform  the  high  intensity,  multi-display  task  used  in  this  study, 
we  believed  that  participants  may  have  come  into  the  experiment  with  varying  levels  of 
experience.  Many  of  the  participants  had  participated  in  previous  versions  of  this  experiment 
and/or  another  experiment  that  was  taking  place  that  involved  playing  a  video  game.  Therefore,  a 
measure  of  ‘Experience’  was  calculated  for  each  participant,  as  follows:  i)  a  point  was  assigned  if 
the  individual  had  participated  in  the  video  game  research  study;  ii)  a  point  was  assigned  for 
participating  in  a  previous  alerting  study,  and  2  points  if  they  reported  participating  in  more  than 
one  alerting  study;  iii)  a  point  was  assigned  for  reporting  some  moderate  experience  with  video 
games  (in  general),  and  2  points  if  the  participant  reported  being  a  heavy  player  of  action  video 
games.  These  points  were  then  summed  to  create  an  ‘Experience’  score  for  each  participant.  The 
maximum  score  any  participant  received  was  three.  In  total,  four  experience  groups  were  created 
with  scores  of  0,  1,  2,  &  3,  with  associated  N’s  equal  to  6,  7,  5,  and  6  respectively. 

3. 1.4.2  Overall  Performance/Training 

Over  the  20  blocks,  each  of  which  was  3  minutes  duration,  participants  showed  improvements  in 
both  the  primary  (alerting)  task  and  secondary  (ship  identification)  task.  By  the  final  block, 
participants’  average  reaction  time  (RT)  to  alerts  was  788  ms,  down  from  898  ms  in  block  1  [F(  1 , 
23)=  7.23,  j?  <  .05],  Likewise,  on  the  secondary  task,  participants  identified  an  average  of  39  ships 
by  the  final  block  compared  to  30  ships  in  the  first  block  [F(  1 ,23)  =  87.591,  p<.001]. 

Not  surprisingly,  the  biggest  leaming/training  effects  appeared  in  the  first  10  blocks,  with  average 
RT  speeding  up  100  ms  in  the  first  10  blocks  [F(  1 ,23 )=8 .20,  p<.0  1  ],  compared  to  only  25  ms 
improvement  in  the  second  10  blocks  [F(  1 ,23)=0.592,  /i=.449], 

3. 1.4.3  Alert  Location  Switching 

In  block  1 1 ,  where  the  alert  location  changed,  the  cost  of  this  location  switch  was  assessed  by 
looking  at  the  difference  in  average  reaction  time  between  block  10  and  subsequent  blocks.  No 
significant  switching  cost  was  evident  in  block  1 1  [F(l,23)=0.1  l,p=.749].  There  did  appear  to  be 
a  cost  in  the  form  of  slower  RTs  a  block  later  in  block  12,  although  this  difference  did  not  reach 
significance  [F(l,23)=2.14,  p=.157]. 
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However,  since  the  trend  in  the  data  was  toward  significance  it  was  deemed  relevant  to  take  into 
account  the  typical  distribution  of  RT  data  (negative  skew)  and  to  conduct  a  mixed-effects 
regression  analysis  using  log-transformed  RT  data  and  conservative  outlier  removal.  The  results 
from  this  more  powerful  analysis  showed  a  difference  between  block  10  and  block  12  that  was 
significant  [ t  =2.34,  p  <  .05],  with  a  mean  RT  cost  of  56  ms  (M= 872  ms  in  block  12  compared  to 
M=  816  ms  in  block  10).  Using  this  approach,  the  difference  between  block  10  and  block  11 
remained  not  significant,  with  /W=822  ms  in  block  1 1  compared  to  M=  816  ms  \t=  -0.09,  p  =  .926], 

3.1. 4.4  Experience 

3.1. 4.4.1  Overall  Experience  Effects 

Using  the  Experience  scores  for  each  participant,  a  correlation  between  Experience  and  alert  RT 
in  the  first  experimental  block  \r  =  -.465,  p  <  .05]  was  revealed,  but  not  the  last  block  [r  =  -.266, 
p=. 209],  There  was  also  a  correlation  between  Experience  and  number  of  ships  identified,  with 
more  Experienced  participants  identifying  more  ships  correctly  in  both  the  first  and  final  blocks  [r 
=  561,  p  <.01  and  r  =.516 , p  <.05,  respectively].  A  repeated  measures  ANOVA  on  all  blocks  with 
Experience  (between-subjects)  revealed  a  significant  main  effect  of  Block  [F  ( 1 9,380)  =  3.52,  p  < 
.001],  but  not  Experience  [F  (1,  20)  =  1.36,  p  =  .284],  Although  the  interaction  between 
Experience  x  Block  (across  all  blocks)  was  significant  [ F  (57,380)  =  1.88,  p  <  .001].  This 
suggested  experience  was  moderating  the  block  effect. 

3.1. 4.4.2  Experience  by  Alert  Location  Switching 

The  results  appeared  to  indicate  that  participants  with  no  previous  task  experience  (experience 
level  0)  showed  the  expected  RT  cost  (slower  RT)  at  block  11  &  12,  see  Figure  4.  On  the  other 
hand,  experienced  participants  seemed  to  show  the  opposite  pattern  with  reduced/faster  RT  at 
block  1 1 . 

A  repeated  measures  ANOVA,  restricting  the  analysis  to  blocks  10  through  12,  with  Experience 
(between-subjects),  revealed  a  significant  interaction  between  Block  x  Experience  [F  (6,  40)  = 
3.64,  p  <  .01].  The  main  effects  of  Block  (10-12)  and  Experience  were  not  significant  [F  (2,  40)  = 
2.04, p  =  .143,  and  A  (3,  20)  =  0.91  ,p  =  .455,  respectively]. 

A  follow-up  analysis  showed  that  for  naive  (experience  level  0)  participants  the  cost  at  block  1 1 
was  marginally  significant  [F  (1,5)=6.20,  p  =  .055]  and  significant  at  block  12  [F  (1,5)  =  7.82,  p 
<  .05],  whereas  for  very  experienced  participants  (experience  level  3),  there  was  no  significant 
cost  of  alert  location  switching  at  block  11  [F  (1,  5)  =  1.06,  p  =  .351]  or  block  12  [F  (1,  5)  = 
2.21,  p  =  .197]. 
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Figure  4:  Effects  of  Experience  on  average  RT  to  alerts.  Experience  level  of  0  (Expert)) 
corresponded  to  participants  who  had  never  participated  in  any  of  the  previous  research 
experiments,  and  had  no  experience  with  video  games.  Note:  Average  (group)  data  shown  in 
following  Figure  (Figure  5). 


3.1. 4.5  Age 

3.1. 4.5.1  Overall  Age  Effects 

Due  to  the  wide  range  in  age  of  participants,  an  age  analysis  was  conducted.  Age  was 
significantly  correlated  with  RT  in  that  older  adults  showed  slower  RT  to  alerts,  with  r  =  .542 ,p  < 
.01  and  r  =  .483,  p  <  .05,  for  the  first  and  last  block  respectively,  see  Figure  5.  Age  was  also 
strongly  correlated  to  the  number  of  correct  ship  identifications,  with  older  adults  identifying 
fewer,  with  r  =  -.769,  p  <  .001,  and  r  =  -.769,  p  <  .001,  for  the  first  and  last  block  respectively.  A 
repeated  measures  ANOVA  on  all  blocks  with  age  group  (between-subjects)  revealed  significant 
main  effects  of  both  Age  and  Block  [F  (1,22)=1 1.14,  p  <  .01;  F  (19,  418)=3.04,  p  <  .001],  with 
faster  RTs  for  younger  participants  and  later  blocks.  The  interaction  between  Age  x  Block  (across 
all  blocks)  was  not  significant  [F(19,  418)  =  0.87, p  <  .623]. 
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Figure  5: 

age  and  under  is  depicted  with  diamonds  (N=12,  and  over  50  years  (N=12)  is  depicted  with 
squares.  The  group  or  average  data  is  shown  as  the  heavy  line  (circles). 

3.1. 4.5.2  Age  by  Alert  Location  Switching 

Splitting  the  participants  into  two  age  groups  (median  split),  a  repeated  measures  ANOVA  on 
blocks  10  through  12  with  Age  (between-subjects)  revealed  no  significant  Block  x  Age 
interaction  [F  (2,44)  =  2.20,  p  =  .123],  However,  this  interaction  was  significant  when  only 
considering  blocks  1 1  &  12  [A  (1,22)  =  4.89,  p  <.05].  The  main  effect  of  age  was  also  significant, 
with  older  adults  associated  with  longer  RT  [F  (1,22)  =  8.95,  p  <  .01].  No  main  effect  of  Block 
was  observed  [F  (1,22)  =  0.41,  p  =  .528],  Follow-up  tests  revealed  that  neither  age -group  showed 
a  significant  block  effect,  however,  the  interaction  and  RT  means  clearly  support  a  trend  towards 
a  location-switching  cost  for  older  participants,  but  not  for  younger  participants. 

3.1 .4.6  Experience  versus  Age 

A  moderately  strong  negative  correlation  existed  between  Age  and  Experience  [r  =  -.641  ,  P  < 
.01],  possibly  due  to  the  fact  that  many  of  the  younger  participants  had  experience  with  video 
games,  and  the  older  participants  did  not.  Exploratory  regression  analyses  were  conducted  to  see 
which  variable  (Age  or  Experience)  was  a  better  predictor  of  the  differential  cost  switching 
effects  observed  between  block  10  &  block  11. 

First,  a  standard  regression  was  conducted  to  predict  switch  cost  (block  11  RT  -  block  10  RT) 
with  both  Age  and  Experience  as  predictors.  The  model  was  significant  with  Age  and  Experience 
together  explaining  26%  of  the  variance  [R2  =  .259]  [A(2,23)  =  3.66,  j?<.05],  Based  on  this 
analysis,  both  Age  and  Experience  appeared  to  be  contributing  approximately  equal  amounts  to 
the  model. 

Subsequently,  a  stepwise  regression  was  conducted  on  the  same  dependent  variable  (switch  cost), 
the  results  showing  that  Experience  was  the  better  predictor  of  the  RT  switch  cost,  explaining 
21%  of  the  variance  \R2  =  .21]  [A  (1,  23)  =  5.93,  /A. 05].  Under  this  analysis  it  appeared  that  Age 
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would  have  not  contributed  significantly  to  the  model  \t  =  1.146,  p  =  .265].  Overall,  these 
analyses  suggested  that  Age  or  Experience  were  strong  predictors  of  the  switch  cost  (or  lack  of) 
between  blocks  1 0  and  1 1 .  Although  the  predictive  power  of  these  two  variables  were  somewhat 
redundant  to  each  other,  Experience  was  a  better  predictor  than  Age. 

3.1.5  Discussion 

The  purpose  of  this  study  was  to  investigate  effects  of  changing  the  location  of  a  visual  alert,  as 
assessed  by  reaction  time  to  the  alerts.  It  was  hypothesized  that  alert  response  time  would  slow 
down  immediately  after  the  location  switch,  and  that  one  explanation  for  this  cost  might  be  that 
the  attentional  system  can  be  trained  to  capture  and  process  relevant  information  even  when  it 
appears  outside  the  area  of  focal  attention.  Consequently,  a  deficit  in  performance  would  be  noted 
when  that  information  appears  in  a  new,  untrained,  location,  as  in  this  experiment  where  the 
location  of  the  alert  changed.  Thus,  even  when  an  operator’s  attention  is  fully  concentrated  on  an 
unrelated  task  and  area  of  the  display,  an  alert  would  be  captured  as  a  feature  within  the 
attentional  window  and  a  drop  in  performance  would  follow  when  the  alert  moved  to  a  new 
location. 

The  overall  results  from  this  experiment  showed  that  this  was  not  the  case,  since  no  significant 
decrement  in  performance  was  observed  immediately  after  the  alert  changed  location. 
Interestingly,  however,  is  the  drop  in  response  time  observed  several  minutes  ( 1  block)  after  the 
switch.  Thus,  it  would  appear  that  participants  were  perhaps  more  vigilant  in  detecting  newly 
located  alerts  right  after  the  location  change,  compared  to  later. 

This  finding  prompted  the  investigators  to  look  more  closely  at  factors  known  to  affect  response 
time.  The  effects  of  age,  for  example,  are  well-documented,  with  older  adults  typically  displaying 
slower  response  times  than  younger  (Elaier,  2005;  Porciatti,  et  al.,  1999;  Rypma  et  al.,  2001). 
Experience  in  video  gaming  is  also  seen  as  a  means  of  improving  performance  in  realms  beyond 
the  gaming  world.  Research  has  shown  that  video  gaming  experts  develop  greater  attentional 
capacity  that  increases  with  game  playing  experience  (Green  &  Bavelier,  2003). 

Perhaps  the  most  interesting  finding  in  this  research  then,  was  that  experience  and  age  were 
strong  moderators  of  the  effect  of  slowed  switching.  In  particular,  highly  experienced  and  young 
individuals  did  not  show  the  expected  location-switching  cost.  On  the  other  hand,  inexperienced 
and  older  participants  were  more  likely  to  show  the  alert-switching  cost  that  was  hypothesized. 

The  results  suggest  that  experience  with  a  specific  task,  or  more  general  video  game  experience, 
can  guard  against  the  cost  associated  with  moving  an  alert  to  a  new,  relatively  untrained  location. 

Thus,  it  is  recommended  that  any  visual  alerting  system  implemented  in  the  future  be  presented 
always  at  the  same  location  on  the  screen.  Or,  if  an  alert  needs  to  be  moved,  or  shown  at  different 
locations,  it  is  important  to  train  operators  at  all  possible  locations  the  alert  may  appear  (to  avoid 
a  reaction  time  cost  at  the  new  location). 
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4  PART  3  -  Eye-Tracking  Experiment 


This  study  was  conducted  at  Dalhousie  University,  Halifax,  Nova  Scotia  using  in-house  eye¬ 
tracking  equipment. 

4.1  Eye-tracking  during  a  3-display  Alerting  Task 

4.1.1  Introduction 

Several  research  studies  have  been  conducted  at  DRDC-Atlantic  using  a  3 -display  alerting  task 
designed  to  simulate  the  kind  of  task  a  naval  radar  operator  might  perform  (Roberts  &  Foster- 
Hunt,  2008;  Roberts,  2008;  Crebolder  &  Beardsall,  2009a;  Crebolder  &  Beardsall,  2009b; 
Crebolder,  Salmon,  Klein,  2010).  The  goal  of  these  studies  has  been  to  determine  what  form 
visual  alerts  should  take  to  be  most  effective  at  capturing  attention.  During  this  3 -display  task 
alerts  have  been  shown  on  one  or  all  of  the  displays  and  the  most  recent  study  investigated 
whether  or  not  reaction  times  to  alerts  varied  as  a  function  of  which  screen  participants  were 
attending.  In  that  study  the  cursor  (mouse)  location  was  used  as  a  proxy/indicator/predictor  of 
where  participants  were  attending  when  the  alert  appeared.  However,  whether  or  not  cursor 
location  was  actually  a  good  indicator  of  where  a  participant’s  eyes  were  looking  was  not  known. 
Thus,  the  relationship  between  eye-location  and  cursor-location  was  explored  in  the  current 
research. 

The  current  research  study  employed  the  use  of  faceLAB  eye -tracking  software 
(www.seeingmachines.com).  This  front-mounted  camera  system  did  not  require  participants 
to  wear  any  apparatus  on  their  head  (like  the  EyeLink  system,  http://www.sr- 
research.com/EL_ll.html),  and  generally,  allowed  for  greater  freedom  of  movement.  In  addition, 
a  number  of  modifications  were  made  to  the  experimental  task  as  compared  to  previous  studies 
(see  Part  2  -  Location  Switch  Experiment  for  a  general  description  of  the  task  and  3-display 
workstation). 

First,  each  participant  completed  6  blocks  in  which  the  arrangement  of  the  tasks  was  different 
every  time;  that  is,  sometimes  the  radar-like  display  would  be  on  the  left  display  and  sometimes 
on  the  right. 

Second,  the  type  of  task  performed  on  the  second  (Status)  display  was  modified  to  better 
differentiate  it  from  the  third  (Reporting)  display.  Specifically  participants  now  had  feature 
check-boxes  on  this  display  instead  of  categorical/classification  boxes  (‘hostile’/’neutral’)  as  in 
previous  research.  With  previous  designs  participants  would  classify  on  the  Status  display  report 
that  classification  on  the  Status  display  and  then  report  again  on  the  Reporting  display.  With  this 
new  way,  participants  would  only  have  to  report  classification  (‘hostile’  or  ‘neutral’)  on  the 
Reporting  Display. 

Third,  the  ship  features  were  designed  so  that  there  was  always  at  least  one  hostile  attribute 
(ensuring  necessary  clicking  on  the  Status  Display)  since  hostile  attributes  were  what  participants 
were  selecting  on  the  Status  Display. 
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Fourth,  the  penalty  for  missing  ships  was  reduced  (from  a  graphic  of  the  ownship  blowing  up)  to 
a  simple  counter  in  the  comer  of  the  display  indicating  the  number  of  missed  ships.  This  last 
change  was  implemented  primarily  to  reduce  eye-tracking  noise  by  maintaining  continuity  of  the 
task  through  the  block.  That  is,  with  the  old  penalty  system,  the  screen  would  freeze  for  an  entire 
task  for  2  s  while  it  showed  the  ship  blowing  up  picture  -  this  might  cause  participants  to  look 
away.  In  the  current  design,  at  a  minimum,  it  would  have  required  throwing  out  eye-tracking  data 
from  this  period. 

Another  change  in  the  current  design  was  that,  instead  of  each  display  being  presented  on  a 
separate  LCD  screen,  all  3  displays  were  presented  on  the  same  Smartboard  side-by-side  (see 
Figures  6).  This  reduced  the  visual  angles  required  of  the  participant  but  allowed  for  better  eye¬ 
tracking  because  the  participants  did  not  have  to  move  their  eyes  and  head  as  much  to  see  all 
displays.  The  participant  also  sat  further  from  the  displays  than  they  had  in  previous  research  (2.3 
m  instead  of  0.5  m). 

These  changes  were  relatively  minor  and  it  was  assumed  that  they  produced  minimal  effects  on 
the  comparative  value  of  this  research. 

A  number  of  objectives  were  set  out  at  the  beginning  of  this  research:  (1)  To  determine  the 
percentage  of  time  participants  were  looking  at  the  same  screen  the  cursor  was  on,  as  a  function 
of  display  task;  (2)  To  determine  whether  the  relationship  between  eye  and  cursor  was  moderated 
by  practice;  (3)  To  determine  if  task-related  display-switching  occurred  in  the  eye  or  cursor  first 
(and  by  how  much). 

4.1.2  Method 

4.1. 2.1  Participants 

Twelve  (5  females,  7  males)  participants  participated  in  this  study.  All  participants  were  recruited 
through  Dalhousie  University,  Halifax,  Nova  Scotia.  The  participants  had  an  average  age  of  25 
years  (range:  18-31  years).  All  participants  were  right-handed  with  normal  or  corrected  to  normal 
vision.  On  average,  these  participants  reporting  playing  4.17  hours  of  video  games  a  week  (range: 
0-30,  SD:  8.53  hrs/wk). 

4.1. 2. 2  Materials 

4.1. 2. 2.1  General  Equipment 

Two  networked  laptops  were  used  to  control  the  experiment.  Laptop  #1  (LI)  was  a  Dell  Latitude 
E6500,  2.40  GHz,  Intel  Core  Duo  with  2.00  GB  of  RAM  running  Windows  Vista  (SP1)  and 
was  used  to  record  the  eye -tracking  data  using  faceLAB  software,  version  5.0 
(www.seeingmachines.com).  It  was  connected  through  FireWire  cables  to  the  faceLAB  cameras. 
Laptop  #2  (L2)  was  a  Toshiba,  2.17  GHz,  Dual  CPU  with  2.00  GB  of  RAM  running  Windows 
Vista  (SPl)  and  was  used  to  present  the  stimuli  to  a  Smartboard  (www.smarttech.com)  using  a 
VGA  connection.  FaceLAB  cameras  were  screwed  to  a  surveying  stand,  and  faceLAB  16  mm 
camera  lenses  were  used.  The  faceLAB  equipment  and  software  were  setup  in  ‘laboratory’  and 
‘precision’  mode  (i.e.  using  a  single  Infra-Red  lens).  The  desk  at  which  the  participant  was  sitting 


20 


DRDC  Atlantic  CR  2010-058 


was  equipped  with  a  mouse  and  keyboard  which  were  connected  to  the  presentation  laptop  (L2) 
using  USB  extension  cables  (1.8  m  long).  Both  laptops  were  connected  to  the  Smartboard  using  3 
VGA  cables  connected  through  a  TruLink  2-port  VGA  Switcher/Extender  that  allowed  quick 
presentation  switching  between  the  two  laptops.  A  standard  USB  cable  and  IP -protocols  were 
used  to  network  the  two  laptops. 

4.1. 2.2.2  Set-up 

FaceLAB  cameras  were  attached  to  a  surveying  stand  positioned  approximately  1.34  m  from  the 
Smartboard  and  standing  102  cm  tall.  The  cameras  were  angled  pointing  up  approximately  10.5 
degrees.  Each  participant  sat  at  a  desk  71  cm  tall  and  60.5  cm  deep.  The  desk  itself  was  placed  on 
a  wooden  stand  1 0  cm  tall  (desk  was  raised  because  the  eye -tracking  system  required  participants 
to  be  above  the  cameras,  and  surveying  stand  was  already  as  low  as  it  could  go).  Participants  sat 
in  an  adjustable  grey  desk  chair  with  no  arms.  The  chair  could  be  adjusted  up  and  down  for  a 
height  range  of  32-45  cm.  The  wheels  were  removed  from  the  chair,  and  the  chair  was  placed  in 
a  wooden  stand  5.5  cm  tall,  so  that  participants’  movements  would  be  minimized.  The  chair  could 
still  rotate  up  to  90  degrees  in  either  direction.  The  chair  itself  put  participants  approximately  90 
cm  from  the  faceLAB  cameras.  The  experimenter  sat  approximately  75  cm  behind  the  participant 
in  front  of  a  second  desk  that  held  both  laptops  as  well  as  a  desktop  computer  that  was  used  for 
data  analysis,  but  was  not  used  during  data  collection.  For  a  better  sense  of  the  set-up  refer  to 
Figures  6A  and  6B  (and  the  additional  photos  in  Annex  C). 


Figure  6:  A.  The  Smartboard  with  the  3  display  task  up  (faceLAB  cameras  shown  in  foreground). 
B.  The  desk,  keyboard  and  mouse  used  by  the  participant  during  the  experiment.  (For  a  better 
sense  of  the  overall  set-up  refer  to  Annex  C). 

4.1. 2.2. 3  Task:  Ship  Identification  and  Alerting 

The  task  involved  shifting  attention  between  three  displays  as  a  part  of  a  ship  classification  task 
that  required  participants  to  determine  whether  a  ship  was  hostile  or  neutral.  The  three  displays 
were  labelled  as  follows:  (1)  Tactical  display,  which  showed  incoming  ships  depicted  as  white 
triangles  on  the  screen.  (2)  Status  display,  contained  information  about  the  ship  selected  on  the 
Tactical  display,  as  well  as  checkboxes  for  hostile  attributes.  (3)  Reporting  display  contained  a 
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textbox  for  reporting  (hostile  or  neutral),  as  well  as  a  submission  (“Enter”)  box,  and  scores  for 
number  of  ships  identified.  A  copy  of  handout  provided  to  aid  instructions  is  shown  in  Annex  D. 

The  participant’s  task  was  to  identify  ships  by  (1)  selecting  a  ship  on  the  tactical  display  by 
clicking  the  mouse  on  the  ship,  (2)  checking  off  all  hostile  ship  attributes  on  the  list  of  neutral  and 
hostile  attributes  that  appeared  on  the  status  display  and  (3)  typing  a  letter  string  on  the  reporting 
display  to  classify  the  ship  as  hostile  (‘asd’),  or  neutral  (‘qwe’)  and,  subsequently  clicking  the 
submit  button.  Ships  with  two  or  more  hostile  attributes  were  correctly  classified  as  hostile,  and 
ships  with  one  hostile  attribute  were  correctly  classified  as  neutral  (every  ship  had  at  least  one 
hostile  attribute  to  ensure  mouse  movement  to  the  status  display  was  required  every  time).  Ship 
attributes  were  chosen  randomly  by  the  computer  but  resulted  in  equal  numbers  of  hostile  and 
neutral  ships.  Ship  attributes  included  Size  (small  or  large),  Speed  (fast  or  slow)  and  Possible 
Weapons  (yes  or  no).  Examples  of  the  handout  used  to  aid  instructions  to  the  participants  shown 
in  Annex  D. 

Correctly  identified  ships  would  disappear  from  the  Tactical  display,  whereas  incorrectly 
identified  ships  remained  on  the  display.  All  ships  gradually  moved  toward  the  centre  circle  that 
represented  the  participants’  ownship.  If  a  ship  reached  the  centre  before  it  was  identified,  the 
ship  would  disappear  and  the  score  for  number  of  missed  ships  would  increase.  Participants  were 
instructed  to  attempt  to  minimize  the  number  of  ships  they  missed  and  maximize  the  number  of 
ships  they  correctly  identified. 

The  Tactical  display  screen  was  updated  every  2  seconds  (meaning  the  ships  moved  in 
incremental  steps  toward  the  centre  every  2  seconds).  Ships  moved  relatively  slowly,  and  it  took 
between  1.5-2  minutes  for  a  single  ship  to  get  from  the  edge  to  the  middle  of  the  display.  Thus, 
ships  moved  an  average  of  8.57  cm/min  or  2.2  degrees/min.  There  were  as  many  as  12 
(maximum)  ships  visible  on  the  tactical  display  at  any  one  time.  Ships  were  re-populated  almost 
immediately. 

Alerts  took  the  form  of  a  static  red  bar,  placed  at  the  top-center  of  the  display  screen.  Alerts  were 
displayed  periodically  (approx.  5.11  alerts/min,  which  equalled  about  26  alerts,  range:  24-28,  per 
block)  on  all  three  displays  simultaneously.  Alerts  remained  on  the  screen  until  participants 
pressed  the  spacebar  in  response.  Responding  to  alerts  was  emphasized  as  primary  importance. 

4.1. 2. 2.4  Visual  Angles 

Each  participant  sat  about  2.24  m  from  the  Smartboard  screen.  The  entire  width  of  the 
Smartboard,  156.5  cm  long  and  117  cm  tall,  subtended  a  visual  angle  of  approximately  34.9 
degrees.  The  3  displays  were  each  49  cm  long  with  a  break  of  2  cm  between  each  screen, 
spanning  a  total  of  151  cm,  or  34.0  degrees  of  visual  angle.  The  displays  were  also  37  cm  tall,  but 
data  was  only  presented  on  approximately  18-32  cm  of  that,  depending  on  the  display  type  (4.6- 
8.1  degrees).  For  example,  the  largest  ring  of  the  tactical  display  had  a  diameter  of  29  cm,  with 
the  smaller  ring  at  16.7  cm. 

The  target  ships  were  2.5  x  2.5  cm  (approx.  0.6  degrees).  The  text  on  the  screen  stood  2  cm 
(approx.  0.5  degrees).The  mouse  cursor  was  2  cm  tall  and  1  cm  wide  (approx.  0.5  by  0.3 
degrees).  The  checkboxes  were  2.5  x  2.5  cm  (approx.  0.6  degrees).  The  red  alerts  were  12  cm 
long  and  2.5  cm  tall,  (approx.  3.1  degrees  by  0.6  degrees). 
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4.1. 2. 3 


Procedure 


4.1. 2. 3.1  Set-up  prior  to  each  participant 

Prior  to  participants  arriving  in  the  lab,  (1)  both  laptops  and  the  Smartboard  were  turned  on,  (2) 
the  network  and  physical  connections  were  tested,  (3)  faceLAB  was  started  and  the  experimenter 
ran  a  calibration  on  himself,  testing  for  eye-tracking  accuracy.  If  accuracy  was  low,  the  cameras 
were  re-focused,  re-levelled,  measured  and  calibrated  using  a  special  calibration  board  designed 
for  the  system.  Step  3  was  then  repeated  until  a  good  calibration  could  be  achieved  on  the 
experimenter;  (4)  the  Smartboard  and  presentation  software  were  tested,  ensuring  that  everything 
was  being  sent  properly  to  the  Smartboard,  (5)  paperwork  was  readied,  and  participants  were 
assigned  a  condition. 

There  were  six  possible  screen  arrangements  (1.  STR,  2.  SRT,  3.  TRS,  4.  TSR,  5.  RST,  6.  RTS 
[where  S  refers  to  Status,  T  to  Tactical,  and  R  to  Reporting,  displays]),  and  participants  were 
assigned  to  one  of  6  random  condition  order  designed  in  Latin  square.  The  orders  were:  a) 
126354,  b)  231465,  c)  342516,  d)  453621,  e)  564132,  and  f)  615243.  Each  participant  was 
assigned  to  one  of  these  orders/condition. 

4.1. 2. 3. 2  Procedure  for  Participants 

After  informed  consent,  each  participant  was  introduced  to  the  eye -tracking  equipment  and  given 
a  demonstration  of  a  good  eye-calibration  (on  the  experimenter).  Participants  were  then  asked  to 
sit  at  the  desk  facing  forward.  The  first  step  involved  adjusting  the  participants  chair  up  and  down 
until  their  head  was  centered  in  the  cameras.  FaceLAB  software  was  used  to  calibrate  the 
participants’  eyes.  The  calibration  process  involved  (1)  automatic  and  manual  calibration  of  the 
participants’  “head  model”  and  (2)  calibration  on  the  board  as  measured  by  the  distance  between 
where  participants  were  looking  and  where  the  cameras  detected  participants’  gaze  with  a  9-point 
calibration.  This  second  step  involved  the  Show  SID  (Screen  Intersection  Display)  between  the 
eyes  and  display  locations  pre -measured  and  set  up  in  the  world  model  prior  to  the  experiment. 
These  two  calibration  steps  (head  model,  and  calibration  to  the  screen)  were  repeated  until  a 
satisfactory  calibration  was  achieved  (defined  by  less  than  1  degree  of  visual  error  on  both  the  left 
and  right  eye).  As  a  part  of  the  calibration  step,  participants  also  filled  out  a  short  demographic 
survey.  In  total,  the  calibration  process  lasted  approximately  30  minutes. 

The  3-display  task  was  then  explained  (see  Appendix  B),  and  each  participant  was  provided  with 
a  practice  session.  The  practice  session  lasted  a  duration  of  4  minutes,  and  screen  arrangement 
was  identical  to  the  arrangement  the  participant  would  see  in  the  first  block.  No  eye-tracking  data 
was  recorded  during  practice,  but  the  experimenter  watched  the  eye-tracking  in  real-time  to 
ensure  that  tracking  accuracy  would  be  satisfactory  once  the  task  commenced. 

Once  practice  was  finished,  participants  began  the  first  of  their  6  regular  blocks.  In  each  block, 
the  arrangement  of  the  3 -displays  was  different.  Each  participant  saw  all  possible  arrangements 
across  the  six  blocks.  Each  block  began  with  an  extra  L2  calibration  that  involved  showing  the 
cursor  as  a  diamond  that  moved  around  the  perimeter  of  the  displays  stopping  at  the  edges, 
comers  &  centre.  During  this  calibration,  participants  were  asked  to  follow  the  diamond  with 
their  eyes.  This  set-up  allowed  for  later  spatial  and  temporal  matching  between  where/when 
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participants  were  looking,  and  where/when  the  eye-tracking  cameras  thought  the  participant  was 
looking.  The  L2  calibration  step  lasted  approximately  1  minute,  with  each  block  of  the  task  for  a 
total  5  minutes.  Thus,  each  block  lasted  a  total  of  6  minutes  duration  and  each  participant  ran  in  6 
blocks.  The  duration  of  the  experimental  blocks,  including  breaks  took  approximately  40  minutes. 

After  all  6  blocks  were  completely  participants  were  debriefed,  allowed  to  ask  any  questions  they 
had,  allowed  to  view  some  of  their  eye -tracking  data,  and  compensated  for  their  time.  The  entire 
experiment  lasted  approximately  1.5  hours. 

4.1.3  Additional  Methodological  Results  /  Details 

4.1. 3.1  Clock  Synchronization,  Network  &  Temporal  Resolution 

Due  to  drift  in  computer  clock  times,  especially  laptop  clock  times,  equivalent  clock  times  on 
each  laptop  was  not  assumed.  Instead,  a  network  was  set-up  between  the  two  computers  so  that 
the  time  on  both  computers  could  be  stored  through  the  duration  of  the  experiment  along  with  the 
output.  Specifically  a  server  was  setup  on  LI  using  Python  that  listened  in  the  background  for 
signals  from  L2.  L2,  as  a  part  of  its  program,  had  client  software  that  sent  a  signal  to  LI  on 
iteration  of  its  loop.  What  was  recorded  on  L2  was  (1)  the  time  the  signal  was  sent,  (2)  the  time 
reported  from  LI,  and  (3)  the  time  the  signal  was  received  back  from  LI.  With  this  information  it 
was  then  possible  to  calculate  any  drift  between  the  two  clock  times  and  line  up  the  eye -tracking 
output  from  LI  with  the  behavioural  data  from  L2. 

LI  had  a  temporal  resolution  of  a  single  refresh  at  60  Hz  or  16.66  milliseconds  (ms).  L2  had  a 
temporal  resolution  that  was  more  variable,  but  usually  better,  with  an  average  of  5-6  ms.  This 
meant  L2  generated  3  times  as  much  data  as  LI.  The  network  speed  (round-trip)  was 
approximately  2  ms,  this  meant  any  time  L2  received  from  LI  occurred  approximately  1  ms 
earlier.  This  information  was  used  to  precisely  synchronize  the  clocks  (post  data-collection) 
between  the  two  output  files.  Because  both  computers  were  keeping  their  own  time  it  was 
impossible  to  line  up  time  points  from  LI  and  L2  exactly,  but  with  the  help  of  an  R-script 
synchronization  with  an  average  offset  (error)  of  approximately  2  ms  was  achieved.  This  meant 
eye-tracking  and  cursor  location  data  could  be  interpreted  with  an  average  error  ±2.19  ms.  The 
worst  match  was  102  ms,  although  errors  this  large  were  rare,  with  a  median  =  1.00  ms  (range:  0- 
102  ms,  SD=3.77  ms),  and  errors  larger  than  10  ms  on  only  4%  of  the  trials. 

4.1. 3. 2  Eye  Calibration  (faceLAB  show  SID) 

The  goal  during  the  show  SID  faceLAB  calibration  step  was  to  achieve  the  lowest  possible 
calibration  error.  In  practice,  the  goal  was  to  achieve  an  error  lower  than  1  degree  of  visual  angle 
for  each  participant  (recall  the  entire  screen  spanned  approximately  34.9  degrees  of  visual  angle 
left  to  right).  Across  all  12  participants  an  average  calibration  error  of  0.79  and  0.78  visual 
degrees  was  achieved  for  the  left  and  right  eye  respectively  (SDs  =  0.26  and  0.33  respectively). 
The  worst  calibration  on  any  one  participant  was  visual  errors  of  1.2  and  1.3  degrees  respectively. 
The  best  calibration  on  any  one  participant  was  0.5  degrees  of  error  on  both  eyes. 
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4.1. 3. 3  L2  eye-tracking  Calibration  to  3  displays  (Green  Screens) 

An  additional  calibration  step  was  included  at  the  beginning  of  each  block.  The  goal  of  this  step 
was  to  determine  upon  which  of  the  three  displays  eye-tracking  quality  was  good  both  spatially 
and  temporally.  On  the  left-most  display  across  all  12  participants  for  the  calibration  step,  the 
eye-tracking  cameras  registered  the  eyes  62.2  %  of  the  time  they  were  expected  to  be  there  (of  a 
theoretical  100%).  This  calibration  was  best  on  the  middle  display  (81.8%)  and  at  about  72.5  % 
on  the  right  display.  Sources  of  error  for  this  calculation  were  likely  due  to  location  changes  of 
the  participant’s  head  position.  Specifically,  if  a  person  was  perfectly  calibrated  to  the 
screen/displays  during  the  show  SID  phase,  then,  if  they  moved  their  head  closer  to  the  cameras 
(leaned  in  during  the  task)  the  projected  eye-positions  would  expand  causing  the  software  to 
project  eye-positions  OFF  the  displays.  Fortunately,  due  to  the  fact  that  the  tasks  were  displayed 
on  different  screens  for  different  blocks,  this  error  was  spread  around  equally,  with  70.8  %, 
72.7%,  and  71.6%  accuracy  on  the  (R)eporting,  (S)tatus,  and  (T)actical  display  respectively 
across  the  entire  experiment.  Also,  the  degree  of  this  error  was  quite  variable  by  participant,  with 
the  best  participant  at  93.3%  and  the  worst  participant  at  41.0  %  (M=71%,  with  seven  participants 
above  70%). 

4.1. 3. 4  Eye-tracking  Display  X-axis  Correction 

An  X-correction  was  applied  to  the  data  as  follows.  The  average  X-position  on  each  display  was 
calculated.  These  average  positions  were  then  used  to  convert  any  OFF-display  locations  to  ON- 
display.  Specifically,  for  all  trials  in  which  the  eye  X-position  was  greater  than  or  equal  to  (>=)  to 
the  average  X-position  of  the  right  display,  the  eye-position  was  changed  to  “Right  Display”.  The 
same  fix  was  applied  to  the  left  display  (all  positions  less  than  or  equal  to  the  average  X  for  left 
were  modified).  The  effect  of  this  correction  is  that  a  number  of  trials  where  the  eye -position 
was  labelled  as  “Smartboard”  or  “Nothing”  were  then  considered  either  “LeftDisplay”  or 
“RightDisplay”,  which  hopefully  corrected  for  the  effect  of  participants  leaning  toward  the 
screen.  This  correction  was  only  applied  to  the  Left  and  Right  Display  (no  X-correction  was 
applied  to  the  Centre  Display). 

This  data  correction  improved  L2  calibration  noticeably  with  cameras  registering  eyes  on  the  left 
display  86.5%  of  the  time  (up  from  62.2%),  on  the  centre  display  81.8%  of  the  time  (same  as 
before),  and  on  the  right  display  82.1%  of  the  time  (up  from  72.5%).  Translated  to  task,  these 
were  accuracies  of  82.5%,  83.9%  and  84.4%  respectively  for  displays  to  be  used  by  R,  S,  and  T. 
This  was  a  noticeable  improvement  which  more  appropriately  equated  eye-tracking  on  all  3- 
displays.  On  a  participant  by  participant  basis,  the  worst  participant  was  brought  up  to  71.9%  (up 
from  41.0%).  The  best  participant  was  still  at  93.3%,  but  the  average  over  the  group  was  83%  (up 
from  71%,  with  all  but  one  participant  over  70%). 

These  percents  should  be  thought  of  as  a  measure  of  the  degree  to  which  the  eye-tracker  was 
correctly  measuring  the  eyes  at  the  position  we  were  asking  participants  to  look.  Thus,  about  80% 
of  the  time  the  eye-tracker  measured  the  eyes  where  we  were  asking  participants  to  look  (during 
the  L2  green  screens  calibration).  Conversely,  20%  of  the  time  the  eye -tracker  was  measuring 
participants’  eyes  at  a  location  other  than  where  we  were  asking  them  to  look.  There  are  multiple 
potential  sources  of  error  here,  (1)  eye-blinks,  when  the  tracker  can’t  find  the  eyes,  (2)  problems 
at  the  boundaries,  meaning  the  corners,  especially  the  upper  right  comers  of  the  left  &  centre 
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display  may  not  have  been  registered  as  ON  that  display,  (3)  participants  not  following  task 
instructions,  including  looking  somewhere  else  or  off  the  screen,  and  (4)  noise  in  the  eye-tracker. 


4.1.4  Primary  (Eye-Tracking)  Results 

4.1. 4.1  Correlation  between  Eye  and  Cursor  During  Task 

The  average  percentage  of  the  time  eyes  and  cursor  were  on  the  same  display  once  the  task  started 
was  84.4%.  This  meant  that  participants’  cursor  and  eyes  were  measured  on  the  same  display 
84.4%  while  performing  the  task,  even  though  this  was  more  than  the  %  of  time  measured  in  the 
calibration  (82.7%  during  L2  calibration).  This  difference  between  task  and  calibration  %  may 
have  been  due  to  the  fact  that  calibration  points  were  at  the  corners  of  the  displays  or  boundaries, 
while  most  task-related  information  was  in  the  centre  of  the  displays.  Still,  this  result  suggests  a 
strong  relationship  between  mouse/cursor  location  and  eye  fixation  location. 

4.1 .4.1.1  By  Task  Screen 

This  cursor-eye  relationship  was  strongest  for  the  Status  and  Tactical  Displays  (86.5%  and  85.4% 
respectively),  and  noticeably  weaker  for  the  Reporting  display  (72.2%).  A  repeated  measures 
ANOVA  with  Block  and  task-display  (Status,  Tactical,  Reporting)  as  predictors  verified  this 
difference  with  a  significant  main  effect  of  task-display,  F(2,l  10)=38.44,  p  <  .001,  but  not  of 
block,  F(5,l  10)=1.65,  p  =  .163.  Also,  the  interaction  between  these  two  variables  did  not  reach 
significance,  F(10,l  10)=  1.66,  p  =  .100,  see  Figure  7. 

Follow-up  comparisons  revealed  that  although  there  was  not  a  difference  between  the  (T)actical 
and  (S)tatus  displays  overall,  there  was  a  significant  difference  in  the  final  block  (block  6),  with  a 
higher  cursor-eye  relationship  for  the  (S)tatus  display,  92.7%  compared  to  86.0%,  F(l,ll)=16.50, 
p  =  .002.  Also,  the  difference  between  the  (T)actical  display  and  (R)eporting  display  (both  of 
which  required  only  a  single  click)  was  diminished  by  the  final  block,  with  86.0%  and  78.8%  in 
the  final  block,  F{\,\\)=6.\6,  p  =  .030. 

Finally,  additional  follow-up  comparisons  (blockl  vs  block6)  revealed  a  trend  for  the  relationship 
between  eye -cursor  to  go  up  over  time.  This  was  most  true  for  the  (S)tatus  and  (R)eporting 
displays,  83.6%  to  92.7%  and  72.5%  to  78. 9%, but  did  not  appear  to  be  true  of  the  (T)actical 
display,  88.9%  to  86.0%.  The  associated  significance  values  for  these  were,  F(l,l  1)=7.48,  p  = 
.019  (S),  F(l,ll)=7.25,/>  =  .021  (R),  and F(  1,1 1)= \.19,p  =  .208  (T),  respectively. 
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Figure  7:  Relationship  between  eye  &  cursor  for  each  of  the  3  task-related  displays  over  the  six 
blocks  (for  all  12  participants  averaged  together).  Error  bars  represent  the  standard  error 
(across  the  12  participants). 

4.1. 4.2  Task-Switching,  Eye  versus  Cursor 

There  was  an  a  prior  hypothesis  that  the  eyes  would  switch  from  display  to  display  faster  than  the 
cursor.  In  other  words,  there  was  an  expectation  that  cursor  display-switching  would  lag  behind 
eye  display-switching.  This  was  tested  by  first  looking  at  each  of  the  3  displays  separately, 
creating  vectors  of  data  for  when  the  participant’s  eyes  were  on  (=1)  versus  off  (=0)  a  display  and 
then  correlating  the  cursor  on/off  vectors  with  the  eye  on/off  vectors.  Next,  the  cursor  data  was 
shifted  both  forward  and  backward  through  time  and  re-correlated  with  the  eye-data.  The  purpose 
of  this  was  to  determine  at  what  time-shift  these  data  were  most  highly  correlated.  The  highest 
correlation  would  be  an  indicator  of  how  much  (on  average)  the  cursor  lagged  ahead  or  behind 
the  eyes. 

The  results  suggested  NO  lag  for  any  of  the  displays,  with  the  highest  correlations  on  all  displays 
for  the  non-time-adjusted  values,  time  =  0,  see  Figure  8.  This  was  true  for  switching  to  any  of  the 
3 -display  types  with  an  average  correlation  r  =  .760,  and  lower  after  a  single  screen  refresh  of 
16.67  ms  (r  =  .759  before  and  r  =  .758  afterward).  These  results  demonstrated  quite  clearly  that 
the  eyes  were  not  ahead  of  the  cursor,  and  the  cursor  was  not  ahead  of  eyes.  Eyes  and  cursor 
switched  display  at  the  same  time  as  each  other,  which  did  not  support  our  a  prior  hypothesis  that 
the  eyes  would  be  a  little  ahead  of  the  cursor. 
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Figure  8:  Correlation  between  lagged  cursor-time  and  eye-time.  High  correlations  indicate  the 
strongest  relationship.  Time  lag  shown  in  seconds  (0.100  sec  =  100  ms).  Correlation  represents 
Pearson-r. 

4.1.5  Behavioral  (Non  eye-related)  Results 

4.1. 5.1  Overall  Performance  -  Primary  Task  (Response  to  Alerts) 

Overall  alert  reaction  time  (RT)  did  not  significantly  improve  (get  faster)  with  practice,  with  a 
mean  alert  RT  of  998  ms  in  block  1  and  912  ms  in  block  6,  F  (1,1 1)=3.30,  p  =  .097.  However, 
with  Experience  added  as  predictor  (between  subjects)  the  interaction  between  Experience  and 
block  was  significant  with,  F  (1,10)  =  5.20,  p  <  .05,  with  inexperienced  participants  showing 
faster  RTs  in  block  6  (1273  ms  in  block  1  compared  to  1080  ms  in  block  6)  and  experienced 
participants  showing  no  difference  RTs  by  block  6  (801  ms  in  block  1  compared  to  792  ms  in 
block  6). 

4.1. 5.1.1  Alert  RT  by  Task  (Cursor  Location) 

Additionally,  alert  RT  as  a  function  of  which  display  the  cursor  was  on  (when  participants 
RESPONDED  to  the  alert)  was  tested  with  a  repeated  measures  ANOVA.  There  was  a  significant 
main  effect  of  cursor  location,  F  (2,  20)  =  10.50,  p  <  .01,  with  the  fastest  RTs  when  the  cursor 
was  on  the  Tactical  Display  (824  ms)  and  slowest  RTs  when  the  cursor  was  on  the  Reporting 
Display  (1018  ms)  with  RTs  on  the  Status  Display  in  between  (913  ms).  Unlike  the  block  effect, 
this  effect  did  not  interact  with  Experience,  F  (2,  20)  =  1.42,  p  =  .266,  that  is,  experienced  and 
inexperienced  participants  both  showed  faster  RTs  to  alerts  when  their  cursor  was  on  the  Tactical 
display  and  slower  RTs  to  alerts  on  the  Reporting  display. 

4.1. 5.2  Overall  Performance  -  Secondary  Task  (#  of  Ships) 

The  secondary  task,  total  number  of  ships  identified  per  (5  minute)  block  was  analyzed.  For 
comparison  to  previous  research  (i.e.  Crebolder,  Salmon,  &  Klein,  2010),  a  measure  of 
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experience  was  included.  Participants  were  classified  as  experienced  if  they  had  participated  in 
another  experiment  using  this  task,  or  reported  playing  more  than  1  hour/week  of  video  games. 
Seven  participants  met  classification  for  ‘experienced’,  while  five  were  considered  to  have  no 
experience. 

Experienced  participants  appeared  to  be  performing  better  at  the  secondary  task,  identifying  an 
average  of  67  ships  compared  to  57  for  non-experienced  participants,  see  Figure  5.  However,  a 
repeated  measures  ANOVA  with  Experience  (between-subjects)  and  Block  (within-subjects) 
revealed  no  significant  Experience  effect,  F  (1,  10)  =  2.66,  p  =  .134,  although  the  Block  effect 
was  significant,  F  (5,  50)  =  31.44,  p  <  .001,  with  more  ship  identifications  in  later  blocks.  The 
interaction  between  Block  by  Experience  was  not  significant,  F (5,  50)  =  1.13 ,p  =  .355,  again  see 
Figure  9. 


1  2  3  4  5  6 

Block 


Figure  9:  Total  Number  of  ships  identified  by  Block.  The  average  for  the  group  (N=12)  is  shown 
in  black.  Experienced  participants  (N=7)  shown  as  triangles,  and  non-experienced  participants 
(N=5)  shown  as  squares.  Error  bars  around  the  average  data  represent  the  standard  error  for  the 
group. 


4.1. 5.3  Exploratory:  Alert  RT  as  a  function  of  Proximity 

An  additional  variable  (Closest  Ship)  was  added  to  data  collection  that  had  not  been  present  in 
previous  paradigms.  This  variable  recorded  the  distance  of  the  ship  closest  to  the  ownship  at  the 
centre  of  the  Tactical  display  (the  ship  closest  to  being  missed).  The  a  prior  hypothesis  about  this 
variable  was  that  although  the  primary  task  was  to  respond  to  alerts,  and  identifying  ships  was  the 
secondary  task,  nonetheless  the  two  would  impact  each  other.  Specifically,  we  predicted  alert  RT 
would  be  slower  when  ships  were  close  to  the  center  of  the  Tactical  display  (a  failure  state  for  the 
secondaiy  task).  Support  for  this  hypothesis  was  found  with  mixed-effects  model  (Experience, 
Block,  Closest  Ship  as  predictors,  and  alert  RT  as  the  dependent  variable),  with  a  significant  main 
effect  of  closest  ship  ( t  =  -2.53)  and  significant  interaction  between  Block  and  closest  ship  (t  = 
2.597).  The  main  effect  supported  the  a  prior  hypothesis  that  participants  would  be  slower  to 
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respond  to  alert  RTs  (primary  task)  when  ships  were  close  to  their  ownship  (secondary  task).  The 
interaction  with  block  suggested  the  size  of  closest  ship  effect  got  smaller  in  later  blocks. 

4.1.6  Discussion 

4. 1.6.1  Cursor  is  a  Good  Proxy  for  Attention  (Eye-Location) 

Eye-location  and  cursor-location  corresponded  highly  with  each  other  during  this  task.  In  fact,  the 
percentage  of  time  eyes  and  cursor  were  together  was  higher  during  the  experimental  task  than 
during  the  L2  calibration.  This  discrepancy  was  likely  due  to  the  fact  that  the  L2  calibration  tested 
the  border  of  the  displays,  while  the  task-relevant  information  occurred  primarily  in  the  centre  of 
the  displays.  This  result,  however,  strongly  suggests  that  the  cursor  is  a  very  strong  predictor  of 
where  the  participant  is  looking.  The  one  task-display  in  which  this  average  correspondence  was 
lower  was  the  Reporting  Display.  The  reduction  for  this  display  may  have  be  due  to  the  fact  that 
the  cursor  task  was  simplest  on  this  display  (click  one  box  which  is  in  the  same  place  every  time) 
in  comparison  to  the  other  displays,  or  due  to  the  fact  that  participants  had  to  type  on  this  display 
(drawing  their  eyes  away  from  the  screen  and  to  the  keyboard).  Although  response  keys 
(‘asd’/’qwe’)  were  chosen  purposely  in  an  effort  to  reduce  the  need  to  look  at  the  keyboard  the 
attempt  may  not  have  been  full  proof.  Certainly,  these  variables  (typing  vs.  simplicity  of  clicking) 
could  be  separated  in  future  research.  All  in  all,  these  results  suggest  the  cursor  is  a  good 
proxy/predictor  of  spatial  attention. 

4.1. 6.2  The  Cursor  becomes  a  Better  Predictor  of  Eye-Location  with  Practice 

For  both  the  Status  and  the  Reporting  display  the  trend  was  towards  a  higher  relationship  between 
participants’  eyes  and  cursor  as  they  got  more  practice.  That  is,  the  eyes  and  cursor  were  more 
likely  to  be  together  on  these  displays  the  better  participants  got  at  the  task.  Interestingly,  this 
relationship  did  not  hold  for  the  Tactical  display. 

4.1. 6.3  Eyes  are  Not  Ahead  of  Cursor 

The  time  lag-analysis  did  not  support  our  prediction  that  the  eyes  would  switch-displays  faster 
than  the  cursor.  Perhaps  participants  were  sufficiently  practiced  at  this  task  that  they  could  plan 
mouse  movements  in  advance  (causing  no  lag).  In  fact,  almost  all  the  participants  (10  of  12) 
allowed  no  ships  to  reach  the  centre  of  the  tactical  display  during  the  duration  of  the  task  (5 
minutes  by  6  blocks  =  30  minutes).  Perhaps  the  participants  were  forced  to  watch  the  cursor 
closely  so  it  would  not  get  lost  on  the  displays.  In  retrospect,  the  cursor  was  mostly  black  on  a 
mostly  black  background,  possibly  making  it  difficult  to  see.  Results  may  have  been  different  if 
the  cursor  had  better  contrast  to  the  background.  However,  this  finding,  although  surprising,  is 
actually  quite  useful  because  it  implies  that  not  only  are  the  eyes  usually  where  the  cursor  is,  but 
that  they  switch  displays/task  at  approximately  the  same  instant. 

4.1 .6.4  Other  factors  Affecting  Alert  RT 

Notably,  a  significant  relationship  was  found  between  reaction  times  to  alerts  (primary  task)  and 
the  distance  of  the  closest  ship  to  the  centrally  located  ownship  (secondary  task).  This  result 
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suggests  that,  when  participants  were  more  in  danger  during  the  secondary  task  (a  ship  about  to 
reach  the  centre),  their  vigilance  on  the  primary  task  dropped  (slower  RTs). 

In  addition,  participants  were  faster  at  responding  to  alerts  when  their  cursor  was  on  the  Tactical 
display,  and  slowest  responding  to  alerts  when  their  cursor  was  on  the  Reporting  display.  Keeping 
in  mind  that  the  location  of  each  display  task  (Tactical,  Status,  Reporting)  was  varied  in  this 
experiment,  a  possible  explanation  for  faster  alert  detection  on  the  Tactical  display  is  that,  if  the 
secondary  task  can  be  thought  of  as  broken  into  trials/events  with  three  steps  [(1)  Tactical  select 
ship,  (2)  Status,  classify  and  click  checkboxes  and  (3)  Report  by  typing  and  clicking]  then 
responses  to  alerts  may  have  been  fastest  between  trials.  In  other  words,  participants  were 
quickest  to  respond  to  alerts  when  they  were  not  already  in  the  process  of  identifying  ships  in  the 
secondary  task.  An  additional  reason  participants  may  have  been  slower  at  the  Reporting  display 
is  that  that  display  required  typing  which  may  have  drawn  participants  attention  away  from  the 
screen. 

4.1. 6.5  Recommendations 

Much  was  learned  from  this  research.  Most  importantly,  cursor-location  is  very  strongly  related  to 
eye-location,  and  the  eyes  do  not  switch  display/task  ahead  of  the  cursor  when  performing  the 
overall  task  used  in  this  study.  These  two  results  together  suggest  that  the  cursor  location,  for 
future  research  using  tasks  like  this,  is  a  more  than  adequate  proxy  for  visual  attention,  and 
concurrent  eye-tracking  during  this  type  of  research  is  likely  unnecessary.  Also,  the  effect  of  the 
secondary  task  (ship  identification)  on  the  primary  task  (responding  to  alerts)  was  better 
characterized  in  the  current  research  than  in  previous,  related  studies.  Specifically,  the  distance 
ships  are  from  the  centre  (in  the  secondary  task)  can  significantly  influence  reaction  times  to 
alerts  (primary  task).  Also,  during  this  task,  participants  were  faster  at  responding  to  alerts  when 
they  were  on  the  Tactical  display  (i.e.  between  events).  Future  research  with  this  task  may  want  to 
more  closely  monitor  the  influence  of  these  secondary  task  variables  on  alert  reaction  times. 
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5  General  Discussion 


In  Part  1 ,  the  most  important  finding  was  that  flashing  does  not  appear  to  matter.  All  flash  rates 
appeared  to  be  responded  to  equally  quickly,  and  there  was  no  difference  between  a  perceived 
50/50  duty  cycle  versus  the  actual  50/50  duty  cycle.  In  part  2,  the  most  important  finding  was  that 
alert  location  switching  does  not  necessarily  result  in  reaction  time  cost  in  the  participants.  The 
appearance  of  a  reaction  time  cost  appeared  to  depend  largely  on  the  participants  previous 
experience  with  the  task  (or  video  games  in  general),  or  the  age  of  the  participant.  Also,  the 
reaction  time  (RT)  delay  appeared  to  be  manifesting  a  block  or  so  later  than  expected.  Finally,  in 
part  3,  the  most  important  finding  was  that  the  cursor  is  a  good  proxy  for  attention.  That  is,  the 
eyes  are  almost  always  where  the  cursor  is,  and  the  two  (eye  and  cursor)  switch  displays  at  the 
same  time.  Also,  this  relationship  appears  to  get  stronger  the  more  time  participants  spend  doing 
the  task. 

A  number  of  secondary  findings  were  also  obtained  from  the  research.  In  part  1,  it  was 
determined  that  alerts  shown  on  the  top  and  bottom  of  the  display  appear  to  be  responded  to  more 
quickly  than  those  on  the  sides.  In  part  2,  the  effect  of  experience  and  age  on  task  proved  to  be 
very  evident,  with  experienced  (and  younger)  participants  identifying  alerts  more  quickly 
(primary  task)  and  identifying  more  ships  (secondary  task).  Experienced  (and  younger) 
participants  also  appeared  to  show  smaller  learning  effects.  In  part  3,  a  new  relationship  between 
RT  to  alerts  and  danger  in  the  secondary  task  (as  measured  by  the  closest  ship  to  the  centre)  was 
identified.  Specifically,  participants  were  slower  responding  to  alerts  when  enemy  ships  where 
close  to  their  ownship.  The  results  also  suggested  faster  alert  RTs  when  participants  were  on  the 
Tactical  screen  (between  events),  and  slower  alert  RTs  when  participants  cursor  was  on  the 
Reporting  display  (at  the  end  of  a  classification  event).  These  findings  are  worth  keeping  in  mind 
during  future  research. 

5.1  Average  RT  to  Respond  to  Alerts 

All  parts  of  this  research  involved  responses  to  flashing  or  static  alerts  with  the  press  of  the  space 
bar.  Not  surprisingly,  participants  RTs  were  fastest  when  they  had  nothing  else  to  do  (i.e.  353  ms 
in  the  single-task  from  Part  1 ,  Experiment  2),  and  only  a  little  bit  slower  when  the  secondary  task 
was  a  simple  mouse-tracking  task  (i.e.  400  ms  in  the  dual-task  from  Part  1,  Experiment  2).  The 
next  slowest  average  RTs  were  obtained  from  the  switch  location  experiment  (i.e.  mean  RTs  = 
832  ms  for  the  entire  group  across  all  blocks),  although  RTs  were  again  faster  for  the  experienced 
participants  in  this  group  ( M  =  753  ms  for  Experience  levels  2  &  3  compared  to  M  =  898  ms  for 
levels  0  &  1).  Finally,  the  slowest  RTs  were  obtained  in  the  eye-tracking  experiment  (Part  3)  with 
M  =  891  ms  (when  considered  RTs  <=  4  seconds).  Again,  RTs  were  faster  for  the  experienced 
participants  compared  to  the  inexperienced  ( M  =  776  ms  compared  to  M  =  1056  ms).  These 
slightly  slower  RTs  in  the  eye -tracking  experiment  may  have  been  related  to  the  fact  that  in  Part  3 
the  alerts  stayed  present  until  they  were  responded  to.  In  contrast,  in  Part  2  the  alerts  would 
disappear  after  4  seconds  if  they  had  not  been  responded  to,  leading  perhaps  to  a  greater  sense  of 
urgency  (and  faster  RTs)  in  Part  2  compared  to  Part  3.  Also,  alerts  appeared  more  regularly  in 
Part  2  (5.33  alerts/min  in  Part  2  compared  to  5.11  alerts/min  in  Part  3)  which  may  have  also 
contributed  to  these  differences.  Practice  effects  may  also  partially  explain  these  differences,  as 
participants  were  performing  the  task  for  longer  in  Part  2. 
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5.2  Summary  of  Primary  Recommendations 


( 1 )  Future  research  can  use  flashing  or  static  alerts  as  there  seems  to  be  no  advantage,  in  terms  of 
RT  to  respond,  for  favouring  one  over  the  other.  (2)  Moving  visual  alerts  to  a  new  location  should 
be  avoided,  unless  participants  have  had  experience  or  training  at  the  new  location.  (3)  Future 
eye -tracking  during  this  task  appears  to  be  unnecessary,  as  the  cursor  location  appears  to  be  a 
good  predictor  of  where  the  eyes  are. 
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Annex  A  Summary  of  Processing  Steps  and  Datafiles 


General  note:  The  program  Python  was  used  to  present  stimuli  in  Part  1  &  Part  3.  This  is  a  free 
program  that  can  be  installed  on  any  computer,  and  the  install  files  for  a  PC  are  provided  in  folder 
‘\Python  &  R  install  files\Python_lnstall’ .  Further,  the  program  R  used  for  many  steps  of  the  data 
processing  and  analysis,  preferentially  over  Excel  and  SPSS.  This  is  because  (1)  R  is  a  free 
program,  (2)  it  can  do  analyses  not  possible  (or  at  least  no  easily  accomplished)  by  Excel/SPSS 
such  as  a  mixed-effects  analysis,  (3)  it  can  handle,  process,  and  analyze  data  more  quickly  and 
efficiently  than  Excel  and  SPSS.  This  last  point  was  especially  important  in  the  case  of  Part  3  in 
which  the  data  files  were  so  large  they  exceeded  the  maximize  size  allowed  for  spreadsheets  in 
Excel  AND  some  of  the  processing  steps  required  hours  of  computing/processing  time  on  an 
optimized  modem  (fast)  computer.  The  files  needed  to  install  R  on  a  PC-computer  are  located  in 
‘YPython  &  R  install  filesVR-project  lnstalT.  Documentation  for  both  Python  and  R  syntax  is 
readily  available  free  of  cost  on  the  internet,  but  1  have  included  an  invaluable  R-resource  the 
Baayen  book  ‘Analyzing  Linguistic  Data:  A  practical  introduction  to  statistics’  in  which  chapter  1 
is  an  introduction  to  R  in  general,  and  chapter  7  to  mixed-effects  models  (‘Baayen2008.pdf). 


Part  1,  Experiment  1.  FlashRate  Research 

1.  Read  all  of  the  data  (.txt)  files  into  Excel  and  save  them  as  excel  documents,  files  in  folder: 
‘PARTl_FlashResearch\FlashResearch_Data\FlashRate_Exp’ 

2.  Sort  the  data  period,  color,  and  position,  then  calculate  the  means  with  a  formula. 

3.  Alternatively,  these  means  can  be  calculated  by  running  the  commands  in  the  R-code: 
FlashRate  Analy.r  -  which  also  contains  the  code  to  run  the  mixed-effects  analysis  on  the 
data. 

4.  Means  were  pooled  together  in  the  Excel  file  ‘FlashRate_GroupData.xls’  and  used  to 
construct  the  figure  for  the  report. 

Part  1,  Experiment  2.  FlashRT  Research 

1.  File  ‘FlashRT_Demographics.xls’  contains  demographic  information  for  participants  #’s  1-19. 
But  only  p6-pl9  had  data  in  which  false  alarm  trials  were  rejected,  so  only  these  were  used. 
Some  of  the  earlier  datafiles  are  stored  in  ‘FlashRTDatalnoFA’  but  the  datafiles  that  were 
analyzed  are  in: 

‘PARTl_FlashResearch\FlashResearch_Data\FlashRT_Exp\R&Analysis_files\FlashRT_Data 

2’ 

2.  R-code  was  used  to  collate  the  data,  and  conduct  the  ANOVAs:  ‘FlashRT_Data2new.r’ 

3.  Analysis  of  Mouse  Error  was  conducted  using  R-code:  ‘FlashRTMouseAnaly.r’ 

4.  Part  of  the  group  data  for  the  Mouse  Error  analysis  is  stored  in  the  file 
‘GroupData_MouseError.xls’.  The  ‘FlashRT  MouseAnaly.r’  file  was  used  to  calculate 
‘_errorSave.txt’  files  for  each  participant  which  is  a  recording  of  average  mouse  error  time 
locked  to  the  onset  of  each  stimulus.  The  Mouse  Error  analysis  in  general  was  not  very 
fruitful,  and  all  of  the  files  are  stored  in: 

‘\PARTl_FlashResearch\FlashResearch_Data\FlashRT_Exp\R&Analysis_files\MouseError_ 

Analysis’ 
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Part  2.  Location  Switch  Experiment. 

Data  Storage 

1 .  All  E-prime  files  and  output  are  contained  in  the  folder: 

‘\PART2_LocSwitch(DRDC)\E-prime  BackUp  Files\Phase4  Jan_2010’ 

The  data  for  participants  in  the  static  condition  are  in: 

‘\PART2_LocSwitch(DRDC)\E-prime  BackUp  Files\Phase4  Jan_2010\Data’ 

The  data  for  participants  in  the  flashing  condition  are  in: 

‘\PART2_LocSwitch(DRDC)\E-prime  BackUp  Files\Phase4  Jan_2010\Cond2flash\Data’ 

E-prime  Changes 

2.  The  primary  change  to  the  program  was  changing  the  number  of  blocks  (to  20),  and  changing 
the  types  of  alerts  per  block  to  meet  the  conditions  (alerts  top,  alerts  bottom,  flashing  or  non¬ 
flashing).  In  the  filenames:  condition  AB  =  top  then  bottom,  BA  =  bottom  then  top,  and  s  = 
static,  f  =  flashing. 

3.  The  program  also  changed  so  that  alerts  would  be  shown  on  all  3  displays  simultaneously  on 
every  trial. 

4.  A  small  bug  in  the  program  was  also  fixed:  alerts  on  the  bottom  right  display  were  being 
drawn  a  few  pixels  in  the  wrong  location  causing  the  alert  removal  to  leave  some  of  the  alert 
up  -  this  was  fixed. 

5.  Practice  condition  was  modified  to  include  a  random  variable  that  would  sometimes  draw  the 
alert  on  the  top  of  the  screen,  and  sometimes  draw  the  alert  on  the  bottom  of  the  screen. 

Data  Processing  (Excel  Document  with  Add-Ins) 

6.  All  data  was  read  into  the  special  Excel  workbook  (parser)  designed  by  Don  Coady  at  DRDC 
Atlantic.  Data  for  the  12  static  condition  participants  were  read  into 
‘Exp4_Static_(N=12)_v03.xls’.  Data  for  the  12  flash  condition  participants  were  read  into 
‘Exp4_Flash_(N=12)_v04.xls’.  These  files  are  stored  in 
‘\PART2_LocSwitch(DRDC)\LS(DRDC)_data_standard_longXLS’ 

7.  The  built-in  SPSS  commands/macros  were  then  run:  ‘Prepare  Data  for  SPSS’  and  ‘Cursor 
Screen  Summary  (Josh)’. 

8.  Smaller  versions  of  these  excel  files  were  then  created  by  deleting  the  data  tabs  and  only 
maintaining  the  summaries  to  create  files  ‘Exp4_Flash_(N=12)_v04_short.xls’  and 
‘Exp4_Static_(N=12)_v03_short.xls’ 

9.  In  both  these  new  files,  a  new  tab  called  ‘RTbyBlock’  was  created,  and  RTs  for  each  of  the  20 
blocks  was  copied  to  this  page  (from  the  summary  tabs)  and  a  simple  figure  was  created. 

Some  values  for  ‘TotalCorrectIDs’  were  also  copied  to  these  tabs. 

10.  These  values  (RTs  per  block  and  number  of  correct  ship  IDs)  were  then  copied  into  SPSS  to 
do  the  repeated  measures  ANOVAs  that  were  reported.  SPSS  files  ‘Exp4_avgRT.sav’  and 
‘Exp4_NumShipCorrect.sav’  respectively.  These  SPSS  files  are  saved  in: 
‘\PART2_LocSwitch(DRDC)\LS(DRDC)_DataAnalysis_Age_Experience’ 

11.  These  RTs  were  then  copied  into  excel  files  ‘DRDC_ExpAnaly_v02.xls’  and 
‘DRDC_AgeAnaly2.xls’  to  create  the  age  and  experience  figures  seen  in  the  report. 
‘DRDC_ExpAnaly_v02.xls’,  in  particular,  contains  additional  information  about  each 
participant,  including  how  the  experience  measure  was  calculated. 

Mixed-Effects  Analysis 

12.  The  mixed-effects  model  was  conducted  in  R  using  the  code  ‘DRDC  EprimeSwitch.r’ 
contained  in:  ‘\PART2_LocSwitch(DRDC)\R_files_&_data’  on  the  full  raw  data  contained  in: 
‘\PART2_LocSwitch(DRDC)\R_files_&_data\DRDC_switchData’ 
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Part  3.  Eye-tracking  Experiment. 


Data  Storage 

1.  The  eye-tracking  data  was  merged  with  the  cursor  location  information  and  stored  as  .csv  files 
in  ‘\PART3_EyeTracking\  JETData&RfilesYEyeTrackingMergedData’,  in  the  fet  folder. 
‘FET’  stands  for  faceLAB  eye-tracking  (data  generated  by  faceLAB),  and  only  data  for 
participants  5-16  were  analyzed.  The  R- files  used  to  construct  these  .csv  files  are  saved  in 
‘\PART3_EyeTracking\  JET_Data&_Rfiles\Key_R-AnalysisFiles\R_MergeDat_Files’.  Note: 
the  faceLAB  data  used  as  input  was  not  provided  (because  these  datafiles  are  huge),  but  the 
R-files  are  provided  as  a  record  to  show  what  processing  steps  were  done  to  create  these. 

2.  The  behavioural-only  data  (cursor  and  alert-responses)  were  saved  as  .txt  files,  and  are  stored 
in  the  JET  folders  in: 

‘\PART3_EyeTracking\  JETData&RfilesYEyeTrackingBehavData’. 

JET  stands  for  ‘Josh  Eye-tracking’  or  ‘Josh  Eye-tracking  behavioural’. 

R-Analysis  Processing  Steps/Files 

3.  R  was  used  for  a  lot  of  the  processing  steps  because  it  handles  big  datafiles  better/faster  than 
Excel  or  SPSS  (some  of  these  processing  steps  required  hours  to  run  on  a  fast  computer).  The 
steps  for  computing  the  ‘correlation  analysis’  or  rather  the  percent  correspondence  between 
cursor  and  eye  described  as  ‘Correlation  between  Eye  and  Cursor  During  Task’  are  present  in 
R-file  ‘ JET  Corre  1  Ana ly_p 0 5 - 1 6 . r ’ . 

4.  The  steps  for  computing  data  necessary  for  the  behavioural  analyses  are  present  in  the  R-file 
‘JET_Behave_p05-16.r’. 

5.  The  file  for  the  time-lag  analysis  described  in  ‘ Task-Switching ,  Eye  versus  Cursor ’  is 
‘JET_AllDat_LagAnaly2.r’  located  in  ‘\PART3_EyeTracking\JET_Data&_Rfiles\Key_R- 
AnalysisFiles’.  The  R-file  generated  the  text  files  located  in 

‘\PART3_EyeTracking\JET_Excel&SPSS\Data_Analysis_XLS_SPSS\JET_LagAnaly’  which 
were  then  combined  to  create  a  figure  for  the  report  in  ‘FET_LagAnaly.xls’ 

6.  The  other  R-files  were  also  used  to  get  descriptive  statistics  for  the  data  that  were  saved  in 
text  files  provided  in  ‘\PART3_EyeTracking\JET_Data&_Rfiles\JET_R_output_TXT  files’ 

Excel  and  SPSS  processing  Steps 

7.  Data  for  the  correlation,  or  %  correspondence  question  were  saved  in 
‘CorrelAnaly_Question.xls’  and  used  to  create  the  figure  for  the  report.  Statistics  was 
conducted  for  this  data  using  SPSS  files  ‘CorrelScreen  Analy.sav’  and 
‘CorrelScreenAnalyCorrectX.sav’  and  the  syntax  file 

‘Syntaxl  CorrelAnaly  rmANOVA.sps’.  The  difference  between  these  two  SPSS  files  is  that 
‘CorrelScreen  AnalyCorrectX.sav’  contains  the  data  post  X-correction  (as  described  in  the 
report)  and  this  was  data  used  in  the  final  analysis.  All  the  file  corresponding  to  this  analysis 
are  location  in: 

‘\PART3_EyeTracking\JET_Excel&SPSS\Data_Analysis_XLS_SPSS\JET_CorrelAnaly’ 

8.  The  SPSS  files  necessary  for  doing  the  remaining  behavioural  (not  eye-tracking)  analyses  on 
the  eye -tracking  data  are  located  in  the  folder 

‘\PART3_EyeTracking\JET_Excel&SPSS\Data_Analysis_XLS_SPSS\JET_BehavData’. 

The  figures  were  again  created  in  the  excel  document,  ‘JET_DataAnaly.xls’. 
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Annex  B  List  of  Folders  and  Variable  Names 


Level  I  folders 
Ethics 

FULLREPORT(EarlierVersions)  - 

PART  lFlashResearch 

PART2_LocSwitch(DRDC) 

PART3_EyeTracking 

Python  &  R  install  files 

SOW  &  Work  Description 


Ethic  protocols  from  the  various  elements  of  this  research. 
Backup  copies  of  earlier  versions  of  the  report. 

All  necessary  files  and  folders  pertaining  to  Part  1 . 

All  necessary  files  and  folders  pertaining  to  Part  2. 

All  necessary  files  and  folders  pertaining  to  Part  3. 

Files  needed  to  install  Python  and/or  R  on  a  PC. 

SOW,  Work  Description  copies 


Part  1  folders  (level  ID 
FlashResearchData 
FlashResearchWriteUp 
PythonBackUpfiles 


-Contains  Python  program.  Data,  and  R-files  for  Exp  1  &  2. 
-Contains  reports  for  Expl  and  Exp  2  (Part  1). 

-Contains  other  versions  of  Python  (stimulus  presentation)  files. 


Part  1  Variable  Names,  Exp  1 

Primary  Data  files  Named:  SUBJ_FlashRate.txt  or  SUBJ_FlashRate.xls 
Variable  Names: 

trial#  -  Trial  number  (order  of  presentation) 

startT  -  Time  the  trial  started  (in  ms  from  start  of  Python  program) 

stopT  -  Time  the  trial  ended  (in  ms  from  start  of  Python  program) 

period  -  The  period  (time  for  complete  cycle)  of  the  flashing  100  =  100  ms,  200  =  200  ms,  etc. 

startOn  -  The  starting  time  ON  for  the  alert  (add  5  ms  to  get  the  ACTUAL  time  on). 

startOFF  -  The  starting  time  OFF  for  the  alert  (period  -  startOn) 

endON  -  The  time  ON  that  the  participant  adjusted  the  alert  to  within  the  trial. 

endOFF  -  The  corresponding  time  OFF  at  the  end  of  the  trial  (period  -  endON) 

color  -  The  color  of  the  alert  (red  or  white) 

position  -  The  position  of  the  alert,  Bot  =  bottom,  Top  =  top,  N  =  near,  F  =  far. 


Part  1  Variable  Names,  Exp  2 

Primary  Data  files  Named:  SUBJ_DATE_TIME_FlashRT.txt 
Variable  Names: 

trial#  -  Trial  number  (order  of  presentation) 

dualTask?  -  Whether  or  not  it  was  the  dualtask  condition  0  =  single,  1  =  dual-task 

tried  startT  -  Time  the  trial  started  (in  ms  from  start  of  Python  program) 

tried  stopT  -  Time  the  trial  ended  (in  ms  from  start  of  Python  program) 

period  -  The  flash  period  (time  for  complete  cycle),  100  =  100  ms,  200  =  200  ms,  etc. 

FlaRateON  -  The  time  the  alert  was  on  (add  5  ms  to  get  the  ACTUAL  time  on). 

FlaRate  OFF  -  The  time  the  alert  was  OFF  (period  -  FlaRateOFF) 

color  -  The  color  of  the  alert  (red  or  white) 

position  -  The  position  of  the  alert,  1-8.  1=  TopFar,  2=BotFar,  3=TopNear,  4=BotNear,  5=LefiFar, 

6=RightFar,  7=LeftNear,  8=RightNear 
AlertON_T  -  The  time  the  alert  came  on  (in  ms  from  start  of  exp) 

ResponseT  -  The  time  the  participant  responded  (in  ms  from  start  of  exp) 

RT  -  The  reaction  time  (RT)  of  the  participant  (AlertON  T  -  ResponseT) 

Fedse _pos  -  Whether  or  not  trial  was  a  false  positive  (whether  response  before  alert  appeared). 
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Part  2  folders  (level  11) 

E-prime  BackUp  Files  -  Copies  of  E-prime  files,  and  data 

LS(DRDC)_data_standard_longXLS  -  Excel  files  of  the  flash  and  static  data  (special  Macro  files) 

LS(DRDC)_DataAnalysis_Age_Experience  -  Age  &  Experience  analysis  Excel  and  SPSS  files 
LS(DRDC)_WriteUp  -  Report  for  Part2  (.doc  files) 

R_files_&_data  -  Datafiles  (in  .txt  format)  and  R-script  for  Mixed  Effects 

analysis 

Recruitment  -  Consent,  debriefing,  instructions  &  other  recruitment  forms 

Part  2  Variable  Names 

Primary  Data  files  Named:  DataJosh-AAS_Phase5_*COND*.txt,  where  *COND*  was  s=static,  f=flash, 
order:  AB  or  BA,  and  then  subject  number. 

Variable  names  in  these  files  were  generated  by  E-prime,  and  were  the  same  as  those  used  in  previous 
experiments. 


Computed  Variables  (in  Excel): 
block#_alarmtype 

Blockl _AvgRT_all  to  Block20_AvgRT_all 

Blockl _Total_Correct_IDs  to  Block20_Total_Correct_IDs 

Cursor _LeftScn,  Cursor _LeftScn,  Cursor  LeftScn 


-  Block  alarm  type  1  or  4=Top,  3  or  6 
=Bottom. 

-  Average  RT  per  block. 

-  The  number  of  correct  ship  IDs  by  block 

-  #  of  times  cursor  on  the  respective  screen 


SPSS  (Exp4_avgRT.sav)  Variable  Names: 


Subj 

age 

Experience 
age  Group 
ageGroup2 
A  l nil  Type 
FlashStatic 
Cond 

Block#_AvgRT_all 
Blckl  1  Cost 


-  Subject  number 

-  Subject  age 

-  Experience  level  of  subject/participant,  from  0  to  3  1 . 

-  First  Age  Grouping  of  participants  (unequal  groups) 

-  Second  age  grouping  of  participants  (median  split  =  equal  Ns) 

-  Alarm  type  (1  or  4=Top,  3  or  6  =Bottom) 

-  Flashing  or  Static  Alert  (s  =  static,  f  =  flashing) 

-  Condition  order  AB  =  Top  then  Bottom,  BA  =  Bottom  then  Top 

-  Average  RT  for  each  block 

-  The  cost  (in  RT)  of  switching  location  {Blockl  1  AvgRT  -  Blockl  1  _AvgRT) 


SPSS  (Exp4_NumShipCorrect.sav)  Variable  Names: 

Subj,  Experience,  age,  ageGroup,  ageGroup2,  AlrmType  -  All  the  same  as  before  (above) 
Block#_Total_Correct_lDs  -  The  number  of  correct  ship  IDs  (secondary  task)  by  block 


Part  3  folders  (level  II) 

EyeTracking  WriteUp  -  Report  for  Part  3,  as  well  instructions,  procedure,  &  storyboard 

JET  Data&  Rfiles  -  Data  and  R-file  used  for  processing  /  pre-processing 

JET  Excel&SPSS  -  Demographics,  Condition  orders,  SPSS  and  Excel  files  (with  figures) 

Part  3  Variable  Names 

Primary  Behavioural  Data  files  Named:  jet##_*COND*_AlertFacelabExp.txt 

Primary  Eye-tracking  Data  files  Named:  zFET_MergData##_*COND*.csv 

. . .  where  ##  =  Subj  Number,  and  *COND*  =  condition  order  (a-f)  and  block  number  (1-6). 


i 


The  values  used  to  compute  these  numbers  are  contained  in  the  file  ‘DRDC_ExpAnaly_v02.xls’ 
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Behavioural  (.txt)  data  Variable  Names  (example  file  name:  ‘jet05_d3_AlertFacelabExp.txt’) 

Type  -  Type  of  trial.  startE  =  start  Experiment,  calibr  =  calibration  phase,  refres  =  regular 

screen  refresh,  mouseP  =  mouse  button  pressed,  moveSH  =  move  ships  on  the  screen, 
alertO  =  alert  ON,  spaceP  =  space  bar  pressed  correctly,  spaceF  =  space  bar  pressed  when 
no  alert  on  screen  (FA),  finish  =  end  experiment. 
pyTime  -  Time  (in  ms)  since  pygame  (Python  module)  started 

GMTs  -  Time  (in  sec)  since  01-01-1970  00:00:00  (Jan  1st,  1970) 

GMTms  -  Time  (in  ms)  over  and  above  GMTms  (0  to  999) 

MouseX  -  Mouse  X  position  on  screen. 

MouseY  -  Mouse  Y  position  on  screen. 

Display  -  Display  mouse/cursor  is  on  9  =  out  of  bounds,  0  =left,  1  ^centre.  2=right. 

Textbox  -  Text  currently  showing  in  the  textbox  ‘ ’  =  nothing  /  blank 

ScrnArrange  -  Arrangement  of  screens  from  left  to  right,  with  S  =  status,  T=tactical,  R=reporting 
Block  -Block  #  (changes  during  calibration  but  kept  constant  for  the  rest  of  the  expeirment) 

Noldent  -  Number  of  correct  ship  identifications  so  far  (secondary  task) 

NoMiss  -  Number  of  ships  missed  (got  to  the  centre  before  identified) 

AlertON  -  Time  an  alert  came  on  (in pyTime  =  ms  since  pygame  started) 

AlertRespT  -  Time  an  alert  was  responded  to  (in pyTime  =  ms  since  pygame  started) 

AlertRT  -  Reaction  time  (RT)  to  alert  (=  AlertRespT  -  AlertON) 

AlertOFF  -  Time  an  alert  went  off  (in  pyTime  =  ms  since  pygame  started) 

tl  -  GMT  time  on  L2  (laptop  #2)  when  ping  was  sent  to  LI  (laptop#l) 

ts  -  GMT  time  reported  from  LI 

t2  -  GMT  time  on  L2  when  ping  was  received  back  from  LI. 

tripT  -  Total  trip/ping  time  (=  t2  -  tl) 

ClosestSH  -  Distance  of  the  ship  closest  to  the  ownship  (absolute  values  of  X  &  Y  distances  added 

together.)  This  variable  was  used  for  an  exploratory  analysis. 

Eye-tracking  (.csv)  data  Variable  Names  (example  file  name:  ‘zFET_MergData05_d4.csv) 

Note:  Variables  ALL  IN  CAPS  come  from  faceLAB  eye-tracking  software. 

FRAME  NUM  -  Number  of  frames  (screen  refreshes)  since  faceLAB  started 

EXPERIMENT  TIME  -  Time  in  ms  since  faceLAB  started  (?) 

GMT  S  -Time  (in  sec)  since  01-01-1 970  00:00:00  (Jan  1st,  1970) 

GMT  MS  -  Time  (in  ms)  over  and  above  GMTms  (0  to  999) 

DELAY  -  **  not  sure 

GAZE  ITEM  NAME  -  The  name  of  the  item  (display)  that  the  participant  was  looking  at 
GAZE  WORLD  X  -  X-location  of  where  person  is  looking 
GAZE  WORLD  Y  -  Y-location  of  where  person  is  looking 

GAZE  WORLD  Z  -  Z-location  of  where  person  is  looking  (depth  in  and  out  of  screen) 

HEAD  ITEM  NAME  -  The  name  of  the  item  (display)  that  the  participant’s  head  was  pointed  at 

HEAD  WORLD  X  -  X-location  where  the  participant’s  head  was  pointing 

HEAD  WORLD  Y  -  Y-location  where  the  participant’s  head  was  pointing 

HEAD  WORLD  Z  -  Z-location  where  the  participant’s  head  was  pointing 

etime  -  Calculated  time  for  merged  with  cursor  data  (=  GMT  S  +  (GMT_MS/ 1000)) 

JBehavTime  -Time  from  behaviourally  data  merged  with  etime  (eye-tracking  time) 

Jtype  -  Type  of  trial  on  behavioural  task  (same  as  above).  Specifically,  calibr  = 

calibration  phase,  refres  =  regular  screen  refresh,  mouseP  =  mouse  button 
pressed,  moveSH  =  move  ships  on  the  screen,  alertO  =  alert  ON,  spaceP  =  space 
bar  pressed  correctly,  spaceF  =  space  bar  pressed  when  no  alert  on  screen  (FA) 
JScnArrange  -  Screen  Arrangement  from  left  to  right,  S  =  Status,  T=Tactical,  R=Reporting 

JMouseX  -  Mouse  (cursor)  X-location 

JMouseY  -  Mouse  (cursor)  Y -location 

JDisplay  -  Display  the  cursor  was  on,  9  =  out  of  bounds,  0=left,  l=centre,  2=  right  display 
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JBlock 

tDiff 

SameScrn2 

SameScrn 


-  Block  Number  (changes  during  calibration,  but  set  =  to  correct  block  #  for  the 
normal  trials) 

-  Difference  between  eye-tracking  time  and  behavioral  time 
-Whether  or  not  eyes  and  cursor  on  the  same  display  (TRUE  or  FALSE) 

-Same  variable  coded  as  numeric  (0=not  on  same  display,  l=same  display) 
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Annex  C  Global  Set-Up  of  Equipment  (Part  3) 


C.1  Eye-tracking  Set-Up:  Participant  &  Experimenter 


Figure  A-10:  Entire  set-up  from  a  distance  (calibration  step).  Participants  were  in  a 
raised  chair  in  front  of  faceLAB  cameras.  Experimenter  at  desk  in  front  of  laptops 
(the  Experimenter’  in  this  photo  was  not  the  actual  experimenter  but  an  RA  posing  as 
the  experimenter). 


C.2  Eye-tracking  Set-Up:  Experimenter  Computers 


Figure  A-ll:  The  two  laptops  used.  LI  on  left  (controlled  faceLAB).  L2  on  right  controlled 
presentation  software  (Python).  Computer  in  back  (LCD)  was  used  for  data  analysis  (but  not 
used  during  the  experiment). 


DRDC  Atlantic  CR  2010-058 


45 


This  page  intentionally  left  blank. 


46 


DRDC  Atlantic  CR  2010-058 


Annex  D  Instructions  for  Task  (Part  3) 


TACTICAL  DISPLAY  (select  a  ship) 


STATUS  DISPLAY  (the  following  are  HOSTILE:  Small,  Fast,  Weapons=Yes) 


Ship  Information:  Hostile: 

Ship  Size  =  Medium  CD 

Ship  Speed  =  Fast  □ 

Possible  Weapons  =  Yes  [jj 


Ship  Information: 

Hostile: 

Ship  Size  =  Medium 

□ 

Ship  Speed  =  Fast 

□ 

Possible  Weapons  =  Yes 
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